BT5-1024x15x32h-rpe-swa-2647500.pb.gz

[peter]

If you run into any problems let me know.

Hey, thanks!
Download of the missing files started by install.cmd was already running on the machine without tensorRt installed, stopped and switched from WLAN to LAN, 3G would have taken too long otherwise, pity the LAN- connected PC gives error:

Installation failed - see the README for alternative download instructions.

Then I just tried to start the binary without running install.cmd at this one PC and with net coming with download of your link, engine did run, probably, because at this one machine I had installed tensorRt sometimes for Ceres way back then. But then changing the net from the default one to the BT5 3700000 gave the already from cuda- Lc0 known error:

Loading weights file from: C:\eng\lc0-v0.32.1-windows-onnx-trt/BT5-1024x15x32h-rpe-swa-3700000.pb.gz
Weights file has multihead format, updating format flag
error RPE weights file not supported.

How to go on now?

[Stephen Ham]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

[Peter Berger]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

Hi Steve,

these nets are usually superior by way of being bigger than their predecessors, which in a +very+ vague way translates to being more knowledgeable.

If you still believe there is any point in playing correspondence chess, you probably want to use one of your computers for such a net/set up to come up with ideas.

You can always blunder-check by running Stockfish for an hour on a position, which already means being completely unbeatable by anything or anybody.

Kind regards.
Peter

[cc2150dx]

If you run into any problems let me know.

Hey, thanks!
Download of the missing files started by install.cmd was already running on the machine without tensorRt installed, stopped and switched from WLAN to LAN, 3G would have taken too long otherwise, pity the LAN- connected PC gives error:

Installation failed - see the README for alternative download instructions.

Then I just tried to start the binary without running install.cmd at this one PC and with net coming with download of your link, engine did run, probably, because at this one machine I had installed tensorRt sometimes for Ceres way back then. But then changing the net from the default one to the BT5 3700000 gave the already from cuda- Lc0 known error:

Loading weights file from: C:\eng\lc0-v0.32.1-windows-onnx-trt/BT5-1024x15x32h-rpe-swa-3700000.pb.gz
Weights file has multihead format, updating format flag
error RPE weights file not supported.

How to go on now?

I'll give you the binary that supports it https://www.mediafire.com/file/wgr724b0 ... 0.exe/file
Change the backend to onnx-trt e.g lc0 -b onnx-trt

[cc2150dx]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

From Discord: THIS MODEL IS WEAKER THAN BT4 AND IS NOT RECOMMENDED FOR PERSONAL USE!
Seeing how BT5 runs at ~half the speed as BT4 you would need multi gpus (RTX 50xx) to compete with BT4

I didn't see @Peter post

[peter]

Change the backend to onnx-trt e.g lc0 -b onnx-trt

Tried this with the newly downloaded binary and UCI- command to change backend- type:

Code: Select all

setoption name Backend value onnx-trt
go depth 9
Found pb network file: C:\eng\lc0-v0.32.1-windows-onnx-trt/BT5-1024x15x32h-rpe-swa-3700000.pb.gz
Weights file has multihead format, updating format flag
Creating backend [onnx-trt]...
...
info depth 9 seldepth 32 time 158224 nodes 57102 score cp 11 nps 1998 tbhits 0 pv d2d4 d7d5 c2c4 c7c6 c4d5 c6d5 c1f4 g8f6 e2e3 b8c6 f1b5 c8g4 g1f3 e7e6 d1a4 g4f3
bestmove d2d4 ponder d7d5

So far so well, with UCI option Backend onnx-trt it runs in Fritz 20 too, thanks again. And yes, I know the BT5 net meant for high end GPU- installations only, so I won't give it much trials at the PC it's already running on now, because this one has single Nvidia 3070ti only. When I'll also succeed to install tensorRt at WLAN- connected newer one too, there's a RTX 5070 installed, but I guess, this will be much too slow still, won't it?

[cc2150dx]

It really just depends on how many threads BT5 is playing against. You just have to find a happy medium. There should be no issues against Reckless, PlentyChess, etc. Stockfish will be the biggest issue.

[peter]

As expected not much difference to be seen in screening- tests like the 256- suite and 30"/pos. between BT4 and BT5- nets with RTX 5070, the older one 3070ti being yet significantly better with BT4 but with BT5. MuPV4 (MultiPV=4) only for fair comparison to A-B-engines with this one setting, even if those profit much more from it in suites with hardware- TC like that of course, 8t=8 threads of 16x4.3GHz CPU:

Code: Select all

    Program                                    Elo   +/-  Matches  Score   Av.Op.   S.Pos.   MST1    MST2   RIndex
  
  8 Stockfish-251112-8t-MuPV4                : 3554    2  38124    57.9 %   3498   189/256    3.4s   10.4s   0.61
 10 Stockfish17.1-8t-MuPV4                   : 3550    2  38999    57.5 %   3498   195/256    4.3s   10.4s   0.55

 13 Lc0v0.32.0-3070ti-1740-MuPV4             : 3547    2  37773    56.9 %   3499   193/256    4.8s   11.0s   0.57

 17 Lc0v0.32.1-RTX5070-6147500PT-MuPV4       : 3543    2  38481    56.4 %   3499   187/256    4.1s   11.0s   0.55

 25 PlentyChess7.0.22-8t-MuPV4               : 3540    2  37664    55.8 %   3499   181/256    3.8s   11.4s   0.54
 26 Lc0v0.32.0-1740-MuPV4-RTX5070            : 3540    2  38376    55.9 %   3499   184/256    4.0s   11.3s   0.52

 48 Lc0v0.31.0onnx-RTX5070-BT5-3700M         : 3532    2  38003    54.8 %   3499   185/256    5.1s   12.0s   0.49

 71 Lc0v0.31.0-dag-onnx-3070ti-BT5-3700M     : 3519    2  37659    52.9 %   3499   175/256    5.4s   13.2s   0.44

MST1  : Mean solution time (solved positions only)
MST2  : Mean solution time (solved and unsolved positions)
RIndex: Score according to solution time ranking for each position

[lucario6607]

As expected not much difference to be seen in screening- tests like the 256- suite and 30"/pos. between BT4 and BT5- nets with RTX 5070, the older one 3070ti being yet significantly better with BT4 but with BT5. MuPV4 (MultiPV=4) only for fair comparison to A-B-engines with this one setting, even if those profit much more from it in suites with hardware- TC like that of course, 8t=8 threads of 16x4.3GHz CPU:

Code: Select all

    Program                                    Elo   +/-  Matches  Score   Av.Op.   S.Pos.   MST1    MST2   RIndex
  
  8 Stockfish-251112-8t-MuPV4                : 3554    2  38124    57.9 %   3498   189/256    3.4s   10.4s   0.61
 10 Stockfish17.1-8t-MuPV4                   : 3550    2  38999    57.5 %   3498   195/256    4.3s   10.4s   0.55

 13 Lc0v0.32.0-3070ti-1740-MuPV4             : 3547    2  37773    56.9 %   3499   193/256    4.8s   11.0s   0.57

 17 Lc0v0.32.1-RTX5070-6147500PT-MuPV4       : 3543    2  38481    56.4 %   3499   187/256    4.1s   11.0s   0.55

 25 PlentyChess7.0.22-8t-MuPV4               : 3540    2  37664    55.8 %   3499   181/256    3.8s   11.4s   0.54
 26 Lc0v0.32.0-1740-MuPV4-RTX5070            : 3540    2  38376    55.9 %   3499   184/256    4.0s   11.3s   0.52

 48 Lc0v0.31.0onnx-RTX5070-BT5-3700M         : 3532    2  38003    54.8 %   3499   185/256    5.1s   12.0s   0.49

 71 Lc0v0.31.0-dag-onnx-3070ti-BT5-3700M     : 3519    2  37659    52.9 %   3499   175/256    5.4s   13.2s   0.44

MST1  : Mean solution time (solved positions only)
MST2  : Mean solution time (solved and unsolved positions)
RIndex: Score according to solution time ranking for each position

You do realize that multipv does nothing for leela besides making it output the moves to uci?

[peter]

You do realize that multipv does nothing for leela besides making it output the moves to uci?

That was true as for about version 0.28 and the nets actual then, from later onwards there are differences in time to solution to be seen again and again, even if (no deterministic output of Lc0 at all, not with single CPU- thread neither, unlike as for A-B-engines) it's always a matter of enough data for statistically significant proof. But take a look at the list above, when I ad two runs of older Lc0- versions and nets MultiPV=1, nr32 and 42 were not in list before but are now:

Code: Select all

    Program                                    Elo   +/-  Matches  Score   Av.Op.   S.Pos.   MST1    MST2   RIndex
  
  8 Stockfish-251112-8t-MuPV4                : 3554    2  38124    57.9 %   3498   189/256    3.4s   10.4s   0.61
 10 Stockfish17.1-8t-MuPV4                   : 3550    2  38999    57.5 %   3498   195/256    4.3s   10.4s   0.55

 13 Lc0v0.32.0-3070ti-1740-MuPV4             : 3547    2  37773    56.9 %   3499   193/256    4.8s   11.0s   0.57

 17 Lc0v0.32.1-RTX5070-6147500PT-MuPV4       : 3543    2  38481    56.4 %   3499   187/256    4.1s   11.0s   0.55

 25 PlentyChess7.0.22-8t-MuPV4               : 3540    2  37664    55.8 %   3499   181/256    3.8s   11.4s   0.54
 26 Lc0v0.32.0-1740-MuPV4-RTX5070            : 3540    2  38376    55.9 %   3499   184/256    4.0s   11.3s   0.52

 32 Lc0v0.32.0-dev-1740-MuPV1                : 3539    2  37103    55.6 %   3499   180/256    4.0s   11.7s   0.57

 42 Lc0v0.32.0-4520-MuPV1                    : 3535    2  37370    55.0 %   3500   180/256    4.6s   12.1s   0.54
 
 48 Lc0v0.31.0onnx-RTX5070-BT5-3700M         : 3532    2  38003    54.8 %   3499   185/256    5.1s   12.0s   0.49

 71 Lc0v0.31.0-dag-onnx-3070ti-BT5-3700M     : 3519    2  37659    52.9 %   3499   175/256    5.4s   13.2s   0.44

Theses two runs both were with older one GPU (3070ti), the more versions and nets I added, the less I did let MuPV1 and MuPV4 run both to see direct comparison, because the differences weren't never ever big compared to A-B MultiPV1 and MultiPV4 at all and got even smaller with newer versions and nets again, now with RTX 5070 even more like that probably, so I could as well have MuPV1 runs for Lc0 only as well as MuPV4 runs only. The reason, I don't delete the runs of little interest is, EloStatTS gets lower error bar with each and every new run in same list, computing Elo and error for each and every old and new run position- and engine- wise again, and I have A-B-engines run with MultiPV4 (if engine supports that) ho-hum. especially SF- clones with internal MultiPV- mode profit much from best of their settings in suites and hardware- TC of that kind, so I like to have Lc0 compared that way now and then too, you see? It's just some kind of list- cosmetics

Full list is 173 runs big in meantime, if pasting in fora, I just copy the parts of interest and the error bars get lower with the bigger number of compared to each other runs, so what, regards