BT5-1024x15x32h-rpe-swa-2647500.pb.gz

[guinhopinda]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

Hi.

The simple answer is "BT5 is probably far superior in postal play." A detailed answer wouldn't fit here, but I recommend the book https://www.amazon.com.br/Xadrez-2022-M ... 6500377001 (it's in Portuguese, but there's an English version). As an ICCF GM, you can get free access through www.sigmasociety.net (see the "admission criteria" page).

I'll summarize the main points to consider:
1. For "normal" engines (pre-A0/Lc0), there's generally a gain of 42 absolute rating points for each doubling in processing speed (or doubling in reflection time with "ponder" disabled).
2. For Lc0, this is different and more like humans, with about 90 to 120 rating points for each doubling. This can vary greatly; in some cases it can be 20 or 30, due to the asymptotic limit determined by excess draws, or 200+ when the time is very short (or processing is very slow) and blunder filtering has a large weight.

This is the basis of the argument. Lc0 gains strength with increased analysis time compared to other engines. Furthermore, more complex networks feel this effect more strongly, while engines with simpler networks more closely resemble "traditional" programs.

However, there are other factors to consider. Training a complex network is much more time-consuming and takes longer to reach optimized parameter weights. This can take months or years, therefore BT5 networks may not yet be as well optimized as BT4 and other smaller networks. The BT5-1024x15x32h-rpe-swa-2647500.pb.gz network, for example, might be weaker than BT4 at almost all time controls, although this might change at postal time controls. The BT5-1024x15x32h-rpe-swa-3700000.pb.gz network is on another level and is almost certainly superior to BT4 at postal time controls.

This is a sufficient summary to clarify the basics, but there are many other subtleties to consider. If you want more details, I highly recommend the book I cited. There is another book that discusses SF and Lc0 training, volume 4 of “Guia dos apodícticos”.

Cheers!

Another point to consider is the 'understanding' of more sophisticated strategic concepts. In the 2017 A0-SF8 matches, it became clear that AlphaZero understands the game much better than SF8, although SF8 calculates concrete lines much better. The correct judgment about 'who is better in a certain position' is fundamental to determining the selection of the best moves in quiet positions (the majority). Again, larger networks are superior in this (for similar architectures, similar optimization times, etc.).

[Rebel]

This link will download a BT5 net directly ... https://plutie.ca/lc0/BT5-1024x15x32h-r ... 0000.pb.gz

I decided to test this BT5 version, 3000 games, hardware RTX 4080 SUPER.

https://prodeo.actieforum.com/t1818p75- ... ults#18809

[lucario6607]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

Hi.

The simple answer is "BT5 is probably far superior in postal play." A detailed answer wouldn't fit here, but I recommend the book https://www.amazon.com.br/Xadrez-2022-M ... 6500377001 (it's in Portuguese, but there's an English version). As an ICCF GM, you can get free access through www.sigmasociety.net (see the "admission criteria" page).

I'll summarize the main points to consider:
1. For "normal" engines (pre-A0/Lc0), there's generally a gain of 42 absolute rating points for each doubling in processing speed (or doubling in reflection time with "ponder" disabled).
2. For Lc0, this is different and more like humans, with about 90 to 120 rating points for each doubling. This can vary greatly; in some cases it can be 20 or 30, due to the asymptotic limit determined by excess draws, or 200+ when the time is very short (or processing is very slow) and blunder filtering has a large weight.

This is the basis of the argument. Lc0 gains strength with increased analysis time compared to other engines. Furthermore, more complex networks feel this effect more strongly, while engines with simpler networks more closely resemble "traditional" programs.

However, there are other factors to consider. Training a complex network is much more time-consuming and takes longer to reach optimized parameter weights. This can take months or years, therefore BT5 networks may not yet be as well optimized as BT4 and other smaller networks. The BT5-1024x15x32h-rpe-swa-2647500.pb.gz network, for example, might be weaker than BT4 at almost all time controls, although this might change at postal time controls. The BT5-1024x15x32h-rpe-swa-3700000.pb.gz network is on another level and is almost certainly superior to BT4 at postal time controls.

This is a sufficient summary to clarify the basics, but there are many other subtleties to consider. If you want more details, I highly recommend the book I cited. There is another book that discusses SF and Lc0 training, volume 4 of “Guia dos apodícticos”.

Cheers!

Or bt5 hasn't finished training and never will finish training. Had premature lr drops, and will never be spsa tuned like bt4. It is also saturated on t80 data. There is nothing to suggest it would be better at correspondence, and in fact many tests show at equal nodes it is still worse while being slower. the only thing it is good at is policy.

[lucario6607]

Yes i would be interested to see your results. You can look at the code, multipv just makes it output more moves to uci. Dragon mcts used history as policy so even that has a policy. It still searches those moves regardless and policy returns a score for every move possible even if you use such a high pv number that maybe some moves don’t get much visits.

I don't think it's a principal difference as for the number of MultiPV- lines, the position I chose of course has an advantage as example, because 1.f4 is by search of many engines ranked as nr. 10 only, so you see the difference to single primary not until MultiPV=10 and then the better for this, because time to depth is getting bigger, time to solution yet getting lower is easier to distinct from quiet positions without such single forced lines and without such big difference in eval. And why, if you see even Dragon's policy help now and then that way, why shouldn't get Lc0- search more visits of tricky to be found and then searched deeper moves as well sooner with MultiPV? And if thats so as for MultiPV=10, why not for MultiPV=2 as well?

So here we go as for output- examples to Salai- study, 3070ti GPU, BT5 3700000- net and onnx- compile from here:

viewtopic.php?p=986146#p986146

direct link:

https://www.mediafire.com/file/wgr724b0 ... 0.exe/file

To not overdraw the number of signs possible in one posting and yet have output- lines of best and lowest ones in MultiPV=10 I pasted first and last of the 10 with ...- lines in between for the lucidity. Each one single try starts with fen of position and engine- name, directly followed by the first one output with correct best move ranked nr.1. Of course I did let run at each trial some longer to see eval getting more clearly a surely winning one and ranking staying stable, but more then one MultiPV=10- output would have become too much signs and or editing,so I hope it's clear and proving well enough like this.

Code: Select all

4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.
...
11/33  0:19   +1.38    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 Kg6 8.Ke5 h5 
                       9.f4 h4 10.f5+ Kh7 11.f6 h3 12.Ke6 h2 
                       13.Lc2+ Kh6 14.f7 (100.913) 4 TB:5 

11/33  0:19   +1.00    1.a3 Kg7 2.Kd7 Ld2 3.f4 Kf6 4.Ld3 h5 
                       5.Le2 h4 6.Lg4 Lc3 7.Lh3 Ld2 8.Kc6 La5 
                       9.f5 Kf7 10.f4 Ke7 11.b4 Lxb4 12.Lf1 La5 (100.913) 4 TB:5 
...
...
...
...
...
...
11/34  0:19   +1.24    1.Lf5 h5 2.Kd7 Ld2 3.Kc6 La5 4.Lh3 Kf7 
                       5.a3 h4 6.f4 Ke7 7.Kb7 Kf7 8.Lg4 Kf6 
                       9.Ka6 Ke7 10.f5 Kf6 11.Lh3 Kf7 
                       12.Kb7 Kf6 (101.289) 4 TB:5 

11/34  0:19   +1.18    1.Ld1 Ld2 2.Kd7 h5 3.f4 h4 4.Lg4 Kg7 
                       5.Kc6 La5 6.a3 Kf6 7.Kb7 Kf7 8.Ka6 Ke7 
                       9.f5 Kf6 10.Lh3 Kf7 11.Kb7 Kf6 (101.289) 4 TB:5 
    
Second run:                           
                       4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -
                       
Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.

...

11/40  0:19   +1.49    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 Kg6 8.Ke5 h5 
                       9.f4 h4 10.f5+ Kh7 11.Ke6 h3 12.f6 h2 
                       13.Lc2+ Kh6 14.f7 (100.344) 4 TB:5 

11/40  0:19   +1.08    1.Ld3 Kg7 2.Kd7 Ld2 3.f4 Kf6 4.Kc6 La5 
                       5.a3 h5 6.Le2 h4 7.Lg4 Ke7 8.f5 Kf7 
                       9.f3 Ke7 10.f4 Kf6 11.Lh3 Kf7 12.f6 Kxf6 
                       13.b4 c4 14.bxa5 (100.344) 4 TB:5 
...
...
...
...
...
...
...
...
11/40  0:19   +1.04    1.Lb1 Kg7 2.Kd7 Ld2 3.f4 Kf6 4.Ld3 Le1 
                       5.Kc6 La5 6.a3 h5 7.Le2 h4 8.Lg4 Ke7 
                       9.f5 Kf7 10.f3 Ke7 11.f4 Kf6 12.Lh3 Kf7 
                       13.f6 Kxf6 14.b4 (100.344) 4 TB:5 

11/41  0:21   +1.23    1.Kd7 Ld2 2.Lf5 h5 3.Lh3 h4 4.Kc6 La5 
                       5.a3 Kg7 6.f4 Kf7 7.Lg4 Ke7 8.f5 Kf7 
                       9.f3 Ke7 10.f4 Kf6 11.Lh3 Kf7 12.f6 Kxf6 
                       13.b4 c4 14.bxa5 (107.663) 3 TB:5 
                       

Third run:

4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.

11/37  0:20   +1.63    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 Kg6 8.Ke5 h5 
                       9.f4 h4 10.f5+ Kh7 11.Ke6 h3 12.f6 h2 
                       13.Lc2+ Kh6 14.f7 (105.483) 4 TB:5 

11/37  0:20   +1.16    1.Ld1 Ld2 2.Kd7 h5 3.f4 h4 4.Lg4 Kg7 
                       5.Kc6 La5 6.a3 Kf6 7.f5 Kg5 8.Lh3 Kf6 
                       9.f4 Kf7 10.b4 Lxb4 11.Lf1 La5 
                       12.Kd5 (105.483) 4 TB:5 
...
...
...
...
...
...
11/38  0:21   +1.23    1.Kd7 Ld2 2.Lf5 h5 3.Kc6 La5 4.Lh3 h4 
                       5.a3 Kg7 6.f4 Kf6 7.Lg4 Ke7 8.f5 Kf6 
                       9.f4 Ke7 10.b4 cxb4 11.axb4 Lxb4 
                       12.Kxb6 Ld2 13.Kb7 (107.457) 4 TB:5 

11/38  0:21   +1.22    1.Lf5 h5 2.Kd7 Ld2 3.Kc6 La5 4.Lh3 Kg7 
                       5.a3 Kf6 6.f4 h4 7.Kc7 Kf7 8.Kc6 Kf6 
                       9.f5 Kf7 10.f4 Kf6 11.Lg4 Ke7 12.b4 cxb4 
                       13.axb4 Lxb4 14.Kxb6 (107.457) 4 TB:5 

So we had 19, 19 and 20" here, and then with MultPV=1: (here I pasted each time last one output- line with wrong move up followed by first one with correct one, to show with smallest amount of lines the least difference in eval, that the engine "gets" the main points of the forced win even with too low eval already, first few moves seem to be more or less proving, e.g 4.b4. Again of course I waited some longer at each trial to see eval going on climbing and best move staingy stable in output.

Code: Select all

4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.

...

16/61  0:38   +1.23    1.Kd7 Ld2 2.Lf5 h5 3.f4 Kf7 4.Kc6 La5 
                       5.Lh3 h4 6.a3 Kf6 7.Kb7 Ke7 8.Ka6 Kf6 
                       9.f5 Ke7 10.Kb7 Kf6 11.Kc6 Kf7 12.b4 Lxb4 
                       13.Lf1 La5 14.Kd5 (193.686) 3 TB:5 

16/61  0:39   +1.78    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 h5 8.Kf4 Kg6 
                       9.Ke5 h4 10.Kf4 Kh5 11.f3 h3 12.Kg3 Kg5 
                       13.Kxh3 Kf4 14.Kg2 (194.817) 3 TB:5 



4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.

...

15/62  0:33   +1.25    1.Kd7 Ld2 2.Lf5 h5 3.Lh3 Kg7 4.Kc6 La5 
                       5.f4 Kf6 6.Kb7 h4 7.Ka7 Kf7 8.a3 Kf6 
                       9.b4 Lxb4 10.Lf1 La5 (165.717) 3 TB:5 

15/62  0:36   +1.81    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 h5 8.Kf4 Kg6 
                       9.Ke5 h4 10.Kf4 Kh5 11.f3 h3 12.Kg3 Kg5 
                       13.Kxh3 Kf4 14.Kg2 (180.291) 3 TB:5 



4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

Engine: Lc0 v0.31.0onnx (0 MB)
von The LCZero Authors.

..

15/64  0:36   +1.23    1.Kd7 Ld2 2.Lf5 h5 3.f4 Kf7 4.Lh3 h4 
                       5.Lg4 Kf6 6.Kc6 La5 7.a3 Ke7 8.Kb7 Kf7 
                       9.Ka6 Kf6 10.Lh3 Kf7 11.f5 Kf6 12.b4 Lxb4 
                       13.Lf1 (178.346) 3 TB:6 

15/64  0:37   +1.81    1.f4 Lxf4 2.Kd7 Ld2 3.Kc6 La5 4.b4 cxb4 
                       5.Lb3+ Kg7 6.Kd5 Kf6 7.Ke4 h5 8.Kf4 Kg6 
                       9.Ke5 h4 10.Kf4 Kh5 11.f3 h3 12.Kg3 Kg5 
                       13.Kxh3 Kf4 14.Kg2 (186.157) 3 TB:6 

So 39, 36 and 37".
Simplified not quite but almost twice as much average time to solution single primary compared to MultiPV=10, isn't it?

Explain to me how your test works in more detail if you would please. You can look at the code, it literally does nothing. I can also send a few different screenshots from lc0 devs saying the exact same thing.

[peter]

Explain to me how your test works in more detail if you would please.

What do you mean by "my test"?
I simply start the GUI (in given trials Shredder 13) then the engine with therefore empty hash and let it run in analysis (go infinite) mode, when best move comes into output (at first place in MultiPV) and stays stabIe there onwards, I copy it. This I did the three times in a row with the starting- position of the Salai- study (link to YACPDB I gave too)

[d]4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -

and the described conditions as for hard- and software, and these three runs (3xMultiPV=10 and 3xMultiPV=1) I pasted, so what?

Or do you mean the EloStatTS- results with the 256 positions? Those are rather well known round here already too, you can download them from ShashChess- hp or I can give you a direct link too, if you want me to.

If you doubt my results, you can simply reproduce them on your own, if you don't want to do so neither, simply stick to what you think to know better without trying

[guinhopinda]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

Hi.

The simple answer is "BT5 is probably far superior in postal play." A detailed answer wouldn't fit here, but I recommend the book https://www.amazon.com.br/Xadrez-2022-M ... 6500377001 (it's in Portuguese, but there's an English version). As an ICCF GM, you can get free access through www.sigmasociety.net (see the "admission criteria" page).

I'll summarize the main points to consider:
1. For "normal" engines (pre-A0/Lc0), there's generally a gain of 42 absolute rating points for each doubling in processing speed (or doubling in reflection time with "ponder" disabled).
2. For Lc0, this is different and more like humans, with about 90 to 120 rating points for each doubling. This can vary greatly; in some cases it can be 20 or 30, due to the asymptotic limit determined by excess draws, or 200+ when the time is very short (or processing is very slow) and blunder filtering has a large weight.

This is the basis of the argument. Lc0 gains strength with increased analysis time compared to other engines. Furthermore, more complex networks feel this effect more strongly, while engines with simpler networks more closely resemble "traditional" programs.

However, there are other factors to consider. Training a complex network is much more time-consuming and takes longer to reach optimized parameter weights. This can take months or years, therefore BT5 networks may not yet be as well optimized as BT4 and other smaller networks. The BT5-1024x15x32h-rpe-swa-2647500.pb.gz network, for example, might be weaker than BT4 at almost all time controls, although this might change at postal time controls. The BT5-1024x15x32h-rpe-swa-3700000.pb.gz network is on another level and is almost certainly superior to BT4 at postal time controls.

This is a sufficient summary to clarify the basics, but there are many other subtleties to consider. If you want more details, I highly recommend the book I cited. There is another book that discusses SF and Lc0 training, volume 4 of “Guia dos apodícticos”.

Cheers!

Or bt5 hasn't finished training and never will finish training. Had premature lr drops, and will never be spsa tuned like bt4. It is also saturated on t80 data. There is nothing to suggest it would be better at correspondence, and in fact many tests show at equal nodes it is still worse while being slower. the only thing it is good at is policy.

As I explained, BT5-1024x15x32h-rpe-swa-3700000.pb.gz tends to be extraordinarily strong with proper hardware and/or suitable time controls, or both. Please, see: https://www.tcec-chess.com/archive.html ... vpt&game=1

[lucario6607]

Hello All,

As the resident dummy, I have some questions:

1) Does compelling evidence exist that this BT5 is superior to BT4-1740? I've looked at the Leela Open Bench reports, but don't understand it.

2) If it's superior, in what way is it superior? As an ICCF GM, I seek analytical quality, rather than better speed chess results.

Thanks in advance,
-Steve-

Hi.

The simple answer is "BT5 is probably far superior in postal play." A detailed answer wouldn't fit here, but I recommend the book https://www.amazon.com.br/Xadrez-2022-M ... 6500377001 (it's in Portuguese, but there's an English version). As an ICCF GM, you can get free access through www.sigmasociety.net (see the "admission criteria" page).

I'll summarize the main points to consider:
1. For "normal" engines (pre-A0/Lc0), there's generally a gain of 42 absolute rating points for each doubling in processing speed (or doubling in reflection time with "ponder" disabled).
2. For Lc0, this is different and more like humans, with about 90 to 120 rating points for each doubling. This can vary greatly; in some cases it can be 20 or 30, due to the asymptotic limit determined by excess draws, or 200+ when the time is very short (or processing is very slow) and blunder filtering has a large weight.

This is the basis of the argument. Lc0 gains strength with increased analysis time compared to other engines. Furthermore, more complex networks feel this effect more strongly, while engines with simpler networks more closely resemble "traditional" programs.

However, there are other factors to consider. Training a complex network is much more time-consuming and takes longer to reach optimized parameter weights. This can take months or years, therefore BT5 networks may not yet be as well optimized as BT4 and other smaller networks. The BT5-1024x15x32h-rpe-swa-2647500.pb.gz network, for example, might be weaker than BT4 at almost all time controls, although this might change at postal time controls. The BT5-1024x15x32h-rpe-swa-3700000.pb.gz network is on another level and is almost certainly superior to BT4 at postal time controls.

This is a sufficient summary to clarify the basics, but there are many other subtleties to consider. If you want more details, I highly recommend the book I cited. There is another book that discusses SF and Lc0 training, volume 4 of “Guia dos apodícticos”.

Cheers!

Or bt5 hasn't finished training and never will finish training. Had premature lr drops, and will never be spsa tuned like bt4. It is also saturated on t80 data. There is nothing to suggest it would be better at correspondence, and in fact many tests show at equal nodes it is still worse while being slower. the only thing it is good at is policy.

As I explained, BT5-1024x15x32h-rpe-swa-3700000.pb.gz tends to be extraordinarily strong with proper hardware and/or suitable time controls, or both. Please, see: https://www.tcec-chess.com/archive.html ... vpt&game=1

wow really, a leela net of that size can be strong at tcec? Would have never guessed!

That literally proves nothing. Bt4 is still better at all tc

[peter]

If you doubt my results, you can simply reproduce them on your own,

I guess I know now, what could cause your problem with reproducing my output- examples to the Salai- position.
Tried console in meantime too and indeed it takes some longer there than it does in GUI to sort the MultiPV- lines according to their eval. If you look at this one run in console:

Lc0 v0.31.0-dag+git.4d98c58 built Jun 22 2025
setoption name Backend value onnx-trt
go
Found pb network file: C:\eng\lc0-v0.32.1-windows-onnx-trt/BT5-1024x15x32h-rpe-swa-3700000.pb.gz
Weights file has multihead format, updating format flag
Creating backend [onnx-trt]...
info depth 1 seldepth 2 time 13628 nodes 2 score cp 7 nps 181 tbhits 0 pv d2d4 d7d5
...
stop
(that to have backed already created when new position is loaded and engine started again)
...
setoption name Threads value 2
setoption name MultiPV value 10
setoption name SyzygyPath value C:\Syzygy\dtz;C:\Syzygy\syzygy;C:\Syzygy\wdl
setoption name NNCacheSize value 1000000
position fen 4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -
go
...
...
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 123 nps 4156 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 122 nps 4156 tbhits 5 multipv 2 pv c2f5 h6h5 e8d7 g5d2 f5h3 g8f7 d7c6 d2a5 a2a3 h5h4 h3g4 f7f6 f3f4 f6e7 c6b7 e7f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 117 nps 4156 tbhits 5 multipv 3 pv c2d1 g5d2 e8d7 h6h5 f3f4 h5h4 d1g4 g8g7 d7c6 d2a5 a2a3 g7f6 c6b7 f6f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 144 nps 4156 tbhits 5 multipv 4 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 f6g6 e4e5 h6h5 f2f4 h5h4 f4f5 g6h7 f5f6 h4h3 e5e6 h3h2 b3c2 h7h6 f6f7 h2h1q f7f8q h6g5 f8f5 g5h4 f5h7 h4g3 h7h1
...
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 123 nps 4141 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 122 nps 4141 tbhits 5 multipv 2 pv c2f5 h6h5 e8d7 g5d2 f5h3 g8f7 d7c6 d2a5 a2a3 h5h4 h3g4 f7f6 f3f4 f6e7 c6b7 e7f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 158 nps 4141 tbhits 5 multipv 3 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 f6g6 e4e5 h6h5 f2f4 h5h4 f4f5 g6h7 f5f6 h4h3 e5e6 h3h2 b3c2 h7h6 f6f7 h6g5 f7f8q h2h1q f8f5 g5h4 f5h7 h4g3
..
...
info depth 11 seldepth 41 time 28112 nodes 114132 score cp 123 nps 4070 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 41 time 28112 nodes 114132 score cp 199 nps 4070 tbhits 5 multipv 2 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4e5 g2g3 e5f5 f3f4 f5f6 g3g4 f6g6 f4f5 g6f6 g4f4 f6e7
...
info depth 12 seldepth 44 time 30055 nodes 121392 score cp 123 nps 3981 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 12 seldepth 44 time 30055 nodes 121392 score cp 220 nps 3981 tbhits 5 multipv 2 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4e5 g2g3 e5f5 f3f4 f5f6 g3g4 f6g6 f4f5 g6f6 g4f4 f6e7 f4g5 e7e8 f5f6 e8f8 g5g6
...
...
info depth 16 seldepth 67 time 57275 nodes 217008 score cp 171 nps 3879 tbhits 8 multipv 1 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4f5 g2g3 f5g5 f3f4 g5f5 g3f3 f5g6 f3g4 g6f6 f4f5 f6g7 g4g5 g7f8 g5g6 f8e7 g6g7 e7d6 f5f6 d6c5 f6f7 c5d4 f7f8q d4d3 f8f2
info depth 16 seldepth 67 time 57275 nodes 217008 score cp 121 nps 3879 tbhits 8 multipv 2 pv e8d7 g5d2 c2f5 h6h5 f3f4 h5h4 d7c8 g8f7 f5h3 f7f6 f4f5 d2c3 c8c7 c3a5 a2a3 f6f7 f2f3 f7f6 c7c6 f6f7 b3b4 a5b4 h3f1 b4a5

And restarted with same commands except the one for MultiPV:
...
info depth 15 seldepth 63 time 47149 nodes 168191 score cp 123 nps 3826 tbhits 5 pv e8d7 g5d2 c2f5 h6h5 d7c6 d2a5 f5h3 h5h4 a2a3 g8g7 f3f4 g7f7 c6c7 f7f6 c7b7 f6g7 b7c6 g7f6 f4f5 f6f7 f2f4 f7e7 f5f6 e7f6 f4f5 f6f7 b3b4 a5b4 h3f1 b4a5
info depth 15 seldepth 63 time 50656 nodes 186443 score cp 179 nps 3832 tbhits 5 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4f5 g2g3 f5g5 f3f4 g5f5 g3f3 f5g6 f3g4 g6f6 f4f5 f6g7 g4g5 g7f8 g5g6 f8e7 f5f6 e7e8 g6g7 e8d7 f6f7 d7d6 g7f6 d6c5 f6e5 c5b5 f7f8q b5a6 f8c8

So indeed in console ranking of 1.f4 in numbers of MultiPV- lines listed takes some longer then it takes to see it getting highest eval. Yet in GUI (depending on which one, tried Arena and Fritz (20) in meantime too, Fritz is even a little faster than Shredder to rank 1.f4 in MultiPV=10 output best according to its eval, pity in Fritz time and depth isn't to be copied in MultiPV- mode) ranking of best line of MultiPV- output as for best eval seems to get sorted faster than in console, in Shredder I copied the output- lines in order of their eval at the soonest one with 1.f4 getting highest eval too, you see?

Anyhow, if you look at evals, you see the advantage of MultiPV=10 over MultiPV=1 here (at that single position) in console as well as in GUI. Lc0v0.31dag-onnxtrt with BT5 3700000 and RTX 5070 (as well as with 3070ti, differences are to be seen with weaker GPU just some clearer) evaluates
1.f4 rather well reproducible gets best eval in depth 11, with single primary in depth 15 or 16, easiest way to see that is in Fritz(20) GUI, can copy- paste output from this one GUI of course too, but no time nor depth- values are stored in copied output. Yet letting suites run in Fritz (which is necessary to let EloStatTS by Frank Schubert evaluate the results) could well give better time to solution- values in MultiPV- mode but in other GUIs (or in console or in command- line tools like e.g. MEA), regards

[lucario6607]

Explain to me how your test works in more detail if you would please.

Or do you mean the EloStatTS- results with the 256 positions? Those are rather well known round here already too, you can download them from ShashChess- hp or I can give you a direct link too, if you want me to.

If you doubt my results, you can simply reproduce them on your own, if you don't want to do so neither, simply stick to what you think to know better without trying

What a great way to discredit yourself, by mentioning shashchess. Funny how its test show it is better than stockfish. You can also simply read the code of the engine

"If you still care, point him to <https://github.com/LeelaChessZero/lc0/b ... /search.cc> where all references to multipv are in the uci info output function."

[lucario6607]

If you doubt my results, you can simply reproduce them on your own,

I guess I know now, what could cause your problem with reproducing my output- examples to the Salai- position.
Tried console in meantime too and indeed it takes some longer there than it does in GUI to sort the MultiPV- lines according to their eval. If you look at this one run in console:

Lc0 v0.31.0-dag+git.4d98c58 built Jun 22 2025
setoption name Backend value onnx-trt
go
Found pb network file: C:\eng\lc0-v0.32.1-windows-onnx-trt/BT5-1024x15x32h-rpe-swa-3700000.pb.gz
Weights file has multihead format, updating format flag
Creating backend [onnx-trt]...
info depth 1 seldepth 2 time 13628 nodes 2 score cp 7 nps 181 tbhits 0 pv d2d4 d7d5
...
stop
(that to have backed already created when new position is loaded and engine started again)
...
setoption name Threads value 2
setoption name MultiPV value 10
setoption name SyzygyPath value C:\Syzygy\dtz;C:\Syzygy\syzygy;C:\Syzygy\wdl
setoption name NNCacheSize value 1000000
position fen 4K1k1/8/1p5p/1Pp3b1/8/1P3P2/P1B2P2/8 w - -
go
...
...
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 123 nps 4156 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 122 nps 4156 tbhits 5 multipv 2 pv c2f5 h6h5 e8d7 g5d2 f5h3 g8f7 d7c6 d2a5 a2a3 h5h4 h3g4 f7f6 f3f4 f6e7 c6b7 e7f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 117 nps 4156 tbhits 5 multipv 3 pv c2d1 g5d2 e8d7 h6h5 f3f4 h5h4 d1g4 g8g7 d7c6 d2a5 a2a3 g7f6 c6b7 f6f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 36 time 26620 nodes 108943 score cp 144 nps 4156 tbhits 5 multipv 4 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 f6g6 e4e5 h6h5 f2f4 h5h4 f4f5 g6h7 f5f6 h4h3 e5e6 h3h2 b3c2 h7h6 f6f7 h2h1q f7f8q h6g5 f8f5 g5h4 f5h7 h4g3 h7h1
...
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 123 nps 4141 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 122 nps 4141 tbhits 5 multipv 2 pv c2f5 h6h5 e8d7 g5d2 f5h3 g8f7 d7c6 d2a5 a2a3 h5h4 h3g4 f7f6 f3f4 f6e7 c6b7 e7f7 b7a6 f7f6 g4h3 f6f7 b3b4 a5b4 a3b4 c5c4 a6b6
info depth 11 seldepth 37 time 26859 nodes 109743 score cp 158 nps 4141 tbhits 5 multipv 3 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 f6g6 e4e5 h6h5 f2f4 h5h4 f4f5 g6h7 f5f6 h4h3 e5e6 h3h2 b3c2 h7h6 f6f7 h6g5 f7f8q h2h1q f8f5 g5h4 f5h7 h4g3
..
...
info depth 11 seldepth 41 time 28112 nodes 114132 score cp 123 nps 4070 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 11 seldepth 41 time 28112 nodes 114132 score cp 199 nps 4070 tbhits 5 multipv 2 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4e5 g2g3 e5f5 f3f4 f5f6 g3g4 f6g6 f4f5 g6f6 g4f4 f6e7
...
info depth 12 seldepth 44 time 30055 nodes 121392 score cp 123 nps 3981 tbhits 5 multipv 1 pv e8d7 g5d2 c2f5 h6h5 f3f4 g8g7 d7c6 d2a5 a2a3 g7f6 f5h3 h5h4 h3d7 f6e7 d7h3 e7f7 f4f5 f7f6 f2f4 f6e7 f5f6 e7f6 b3b4 c5c4 b4a5 c4c3
info depth 12 seldepth 44 time 30055 nodes 121392 score cp 220 nps 3981 tbhits 5 multipv 2 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4e5 g2g3 e5f5 f3f4 f5f6 g3g4 f6g6 f4f5 g6f6 g4f4 f6e7 f4g5 e7e8 f5f6 e8f8 g5g6
...
...
info depth 16 seldepth 67 time 57275 nodes 217008 score cp 171 nps 3879 tbhits 8 multipv 1 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4f5 g2g3 f5g5 f3f4 g5f5 g3f3 f5g6 f3g4 g6f6 f4f5 f6g7 g4g5 g7f8 g5g6 f8e7 g6g7 e7d6 f5f6 d6c5 f6f7 c5d4 f7f8q d4d3 f8f2
info depth 16 seldepth 67 time 57275 nodes 217008 score cp 121 nps 3879 tbhits 8 multipv 2 pv e8d7 g5d2 c2f5 h6h5 f3f4 h5h4 d7c8 g8f7 f5h3 f7f6 f4f5 d2c3 c8c7 c3a5 a2a3 f6f7 f2f3 f7f6 c7c6 f6f7 b3b4 a5b4 h3f1 b4a5

And restarted with same commands except the one for MultiPV:
...
info depth 15 seldepth 63 time 47149 nodes 168191 score cp 123 nps 3826 tbhits 5 pv e8d7 g5d2 c2f5 h6h5 d7c6 d2a5 f5h3 h5h4 a2a3 g8g7 f3f4 g7f7 c6c7 f7f6 c7b7 f6g7 b7c6 g7f6 f4f5 f6f7 f2f4 f7e7 f5f6 e7f6 f4f5 f6f7 b3b4 a5b4 h3f1 b4a5
info depth 15 seldepth 63 time 50656 nodes 186443 score cp 179 nps 3832 tbhits 5 pv f3f4 g5f4 e8d7 f4d2 d7c6 d2a5 b3b4 c5b4 c2b3 g8g7 c6d5 g7f6 d5e4 h6h5 e4f4 f6g6 f4e5 h5h4 e5f4 g6h5 f2f3 h4h3 f4g3 h5g5 g3h3 g5f4 h3g2 f4f5 g2g3 f5g5 f3f4 g5f5 g3f3 f5g6 f3g4 g6f6 f4f5 f6g7 g4g5 g7f8 g5g6 f8e7 f5f6 e7e8 g6g7 e8d7 f6f7 d7d6 g7f6 d6c5 f6e5 c5b5 f7f8q b5a6 f8c8

So indeed in console ranking of 1.f4 in numbers of MultiPV- lines listed takes some longer then it takes to see it getting highest eval. Yet in GUI (depending on which one, tried Arena and Fritz (20) in meantime too, Fritz is even a little faster than Shredder to rank 1.f4 in MultiPV=10 output best according to its eval, pity in Fritz time and depth isn't to be copied in MultiPV- mode) ranking of best line of MultiPV- output as for best eval seems to get sorted faster than in console, in Shredder I copied the output- lines in order of their eval at the soonest one with 1.f4 getting highest eval too, you see?

Anyhow, if you look at evals, you see the advantage of MultiPV=10 over MultiPV=1 here (at that single position) in console as well as in GUI. Lc0v0.31dag-onnxtrt with BT5 3700000 and RTX 5070 (as well as with 3070ti, differences are to be seen with weaker GPU just some clearer) evaluates
1.f4 rather well reproducible gets best eval in depth 11, with single primary in depth 15 or 16, easiest way to see that is in Fritz(20) GUI, can copy- paste output from this one GUI of course too, but no time nor depth- values are stored in copied output, regards

I get like 1nps with bt5 so not going to try lol. There is a t3d rpe net tho