annoying behaviour of stockfish in analysis

Discussion of chess software programming and technical issues.

Moderators: hgm, chrisw, Rebel

User avatar
RubiChess
Posts: 639
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: annoying behaviour of stockfish in analysis

Post by RubiChess »

Here's the output of Rubi depth 10 search in startpos:

Code: Select all

go depth 10
info depth 1 seldepth 2 multipv 1 time 13 score cp 21 nodes 20 nps 1507 tbhits 0 hashfull 0 pv d2d4
info depth 2 seldepth 3 multipv 1 time 13 score cp 36 nodes 43 nps 3153 tbhits 0 hashfull 0 pv d2d4 g8f6
info depth 3 seldepth 4 multipv 1 time 14 score cp 0 nodes 157 nps 10869 tbhits 0 hashfull 0 pv g2g3 g8f6 f1g2
info depth 4 seldepth 5 multipv 1 time 14 score cp 32 nodes 203 nps 13710 tbhits 0 hashfull 0 pv g2g3 g8f6 f1g2 b8c6
info depth 5 seldepth 6 multipv 1 time 16 score cp 34 nodes 439 nps 27192 tbhits 0 hashfull 0 pv d2d4 g8f6 e2e3 b7b6
info depth 6 seldepth 7 multipv 1 time 17 score cp 34 nodes 658 nps 37914 tbhits 0 hashfull 1 pv d2d4 g8f6 g2g3 c7c6 g1f3 d8a5 b1d2
info depth 7 seldepth 9 multipv 1 time 19 score cp 40 lowerbound nodes 1186 nps 59368 tbhits 0 hashfull 1 pv e2e4
info depth 7 seldepth 7 multipv 1 time 20 score cp 33 nodes 1323 nps 63749 tbhits 0 hashfull 1 pv e2e4 d7d6 b1c3 g8f6 g1f3 e7e5
info depth 8 seldepth 9 multipv 1 time 23 score cp 27 upperbound nodes 1840 nps 77293 tbhits 0 hashfull 2 pv e2e4 e7e5
info depth 8 seldepth 11 multipv 1 time 31 score cp 22 upperbound nodes 3013 nps 96511 tbhits 0 hashfull 2 pv e2e4 e7e5
info depth 8 seldepth 11 multipv 1 time 35 score cp 17 nodes 4025 nps 112061 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 b1c3 c6d4 f3d4 e5d4
info depth 9 seldepth 9 multipv 1 time 36 score cp 17 nodes 4200 nps 113949 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 b1c3 c6d4
info depth 10 seldepth 12 multipv 1 time 41 score cp 23 lowerbound nodes 4669 nps 112036 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 42 score cp 29 lowerbound nodes 4784 nps 112945 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 43 score cp 36 lowerbound nodes 4959 nps 114800 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 43 score cp 44 nodes 5158 nps 117785 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 g8f6 f3e5 f6e4 d2d4 d7d5
bestmove e2e4 ponder e7e5

And the same search displaying every new best move (additional lines have 'time 1000'):

Code: Select all

go depth 10
info depth 1 seldepth 2 multipv 1 time 1000 score cp -11 nodes 1 nps 1000 tbhits 0 hashfull 0 pv a2a3
info depth 1 seldepth 2 multipv 1 time 1000 score cp -9 nodes 2 nps 1000 tbhits 0 hashfull 0 pv b2b3
info depth 1 seldepth 2 multipv 1 time 1000 score cp -2 nodes 3 nps 1000 tbhits 0 hashfull 0 pv c2c3
info depth 1 seldepth 2 multipv 1 time 1000 score cp 6 nodes 5 nps 1000 tbhits 0 hashfull 0 pv e2e3
info depth 1 seldepth 2 multipv 1 time 1000 score cp 10 nodes 11 nps 1000 tbhits 0 hashfull 0 pv c2c4
info depth 1 seldepth 2 multipv 1 time 2 score cp 21 nodes 20 nps 8253 tbhits 0 hashfull 0 pv d2d4
info depth 2 seldepth 3 multipv 1 time 2 score cp 36 nodes 43 nps 15923 tbhits 0 hashfull 0 pv d2d4 g8f6
info depth 3 seldepth 4 multipv 1 time 1000 score cp -2 nodes 52 nps 1000 tbhits 0 hashfull 0 pv d2d4 g8f6 c1g5
info depth 3 seldepth 4 multipv 1 time 3 score cp 0 nodes 157 nps 42489 tbhits 0 hashfull 0 pv g2g3 g8f6 f1g2
info depth 4 seldepth 5 multipv 1 time 4 score cp 32 nodes 203 nps 49140 tbhits 0 hashfull 0 pv g2g3 g8f6 f1g2 b8c6
info depth 5 seldepth 6 multipv 1 time 1000 score cp 11 nodes 282 nps 1000 tbhits 0 hashfull 0 pv g2g3 b8c6 d2d3 g8f6
info depth 5 seldepth 6 multipv 1 time 5 score cp 34 nodes 439 nps 77201 tbhits 0 hashfull 0 pv d2d4 g8f6 e2e3 b7b6
info depth 6 seldepth 7 multipv 1 time 7 score cp 34 nodes 658 nps 92129 tbhits 0 hashfull 1 pv d2d4 g8f6 g2g3 c7c6 g1f3 d8a5 b1d2
info depth 7 seldepth 9 multipv 1 time 1000 score cp 29 nodes 875 nps 1000 tbhits 0 hashfull 1 pv d2d4 g8f6
info depth 7 seldepth 9 multipv 1 time 14 score cp 40 lowerbound nodes 1186 nps 81470 tbhits 0 hashfull 1 pv e2e4
info depth 7 seldepth 7 multipv 1 time 16 score cp 33 nodes 1323 nps 80570 tbhits 0 hashfull 1 pv e2e4 d7d6 b1c3 g8f6 g1f3 e7e5
info depth 8 seldepth 9 multipv 1 time 20 score cp 27 upperbound nodes 1840 nps 91390 tbhits 0 hashfull 2 pv e2e4 e7e5
info depth 8 seldepth 11 multipv 1 time 27 score cp 22 upperbound nodes 3013 nps 108003 tbhits 0 hashfull 2 pv e2e4 e7e5
info depth 8 seldepth 11 multipv 1 time 35 score cp 17 nodes 4025 nps 114727 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 b1c3 c6d4 f3d4 e5d4
info depth 9 seldepth 9 multipv 1 time 37 score cp 17 nodes 4200 nps 112908 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 b1c3 c6d4
info depth 10 seldepth 12 multipv 1 time 41 score cp 23 lowerbound nodes 4669 nps 113052 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 44 score cp 29 lowerbound nodes 4784 nps 108159 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 46 score cp 36 lowerbound nodes 4959 nps 106461 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 47 score cp 44 nodes 5158 nps 108438 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 g8f6 f3e5 f6e4 d2d4 d7d5
bestmove e2e4 ponder e7e5
I must admit that it is far less than i was expecting. So it seems indeed reasonable to show these lines, at least at bigger depth.
Last edited by RubiChess on Sat Nov 09, 2024 6:24 pm, edited 1 time in total.
User avatar
RubiChess
Posts: 639
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: annoying behaviour of stockfish in analysis

Post by RubiChess »

Uri Blass wrote: Sat Nov 09, 2024 4:14 pm [d]8/1ppk4/2q1q2R/1ppp4/8/1P1P4/1PPPR3/4K3 w - - 0 1

Stockfish_24103109_x64_avx2:
Available processors: 0-7
Using 1 thread
info string NNUE evaluation using nn-1cedc0ffeeee.nnue (133MiB, (22528, 3072, 15, 32, 1))
info string NNUE evaluation using nn-37f18f62d772.nnue (6MiB, (22528, 128, 15, 32, 1))
1/9 00:00 40 20k +1.11 Rh6-h7+ Qe6-e7 Rh7xe7+ Kd7-d6
2/6 00:00 276 138k -0.13 Re2xe6 Qc6xe6+ Rh6xe6 Kd7xe6
3/5 00:00 357 179k -0.13 Re2xe6 Qc6xe6+ Rh6xe6 Kd7xe6
4/5 00:00 525 263k -0.22 Re2xe6 Qc6xe6+ Rh6xe6 Kd7xe6
5/6 00:00 734 245k -0.22 Rh6xe6 Qc6xe6 Re2xe6 Kd7xe6 Ke1-e2
This is what's happening:

Code: Select all

info depth 4 currmove e2e6 currmovenumber 1
Score: -83
info depth 2 currmove h6e6 currmovenumber 2
Score: -83
...
info depth 4 seldepth 5 multipv 1 score cp -22 nodes 525 nps 16 hashfull 0 tbhits 0 time 32294 pv e2e6 c6e6 h6e6 d7e6

info depth 5 currmove e2e6 currmovenumber 1
Score: -83
info depth 5 currmove h6e6 currmovenumber 2
Score: -81
...
info depth 5 seldepth 6 multipv 1 score cp -22 nodes 734 nps 22 hashfull 0 tbhits 0 time 32332 pv h6e6 c6e6 e2e6 d7e6 e1e2
The second transposing line gets a slightly better score at depth 5, probably by different move ordering. I don't see a reason to ignore that and keep the old pv.
User avatar
hgm
Posts: 28265
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: annoying behaviour of stockfish in analysis

Post by hgm »

RubiChess wrote: Sat Nov 09, 2024 6:01 pm Here's the output of Rubi depth 10 search in startpos:

Code: Select all

go depth 10
...
info depth 10 seldepth 12 multipv 1 time 41 score cp 23 lowerbound nodes 4669 nps 113052 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 44 score cp 29 lowerbound nodes 4784 nps 108159 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 46 score cp 36 lowerbound nodes 4959 nps 106461 tbhits 0 hashfull 3 pv e2e4
info depth 10 seldepth 10 multipv 1 time 47 score cp 44 nodes 5158 nps 108438 tbhits 0 hashfull 3 pv e2e4 e7e5 g1f3 g8f6 f3e5 f6e4 d2d4 d7d5
bestmove e2e4 ponder e7e5
I must admit that it is far less than i was expecting. So it seems indeed reasonable to show these lines, at least at bigger depth.
Well, it is actually a lot more than I had expected. I don't think it is very useful to print the lower bounds. Exact scores, having a full PV, are quite useful in analysis, though. As long as it doesn't switch PV is would only print a single info line.
RubiChess wrote: Sat Nov 09, 2024 6:21 pmThis is what's happening:

Code: Select all

info depth 4 currmove e2e6 currmovenumber 1
Score: -83
info depth 2 currmove h6e6 currmovenumber 2
Score: -83
...
info depth 4 seldepth 5 multipv 1 score cp -22 nodes 525 nps 16 hashfull 0 tbhits 0 time 32294 pv e2e6 c6e6 h6e6 d7e6

info depth 5 currmove e2e6 currmovenumber 1
Score: -83
info depth 5 currmove h6e6 currmovenumber 2
Score: -81
...
info depth 5 seldepth 6 multipv 1 score cp -22 nodes 734 nps 22 hashfull 0 tbhits 0 time 32332 pv h6e6 c6e6 e2e6 d7e6 e1e2
The second transposing line gets a slightly better score at depth 5, probably by different move ordering. I don't see a reason to ignore that and keep the old pv.
Indeed, you would not want to keep the old PV if you find a better one. But you could have printed it before discarding it, or in fact together with the -83 score. From the POV of an analysis user, this is indeed very annoying behavior.
User avatar
RubiChess
Posts: 639
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: annoying behaviour of stockfish in analysis

Post by RubiChess »

hgm wrote: Sat Nov 09, 2024 7:40 pm But you could have printed it before discarding it, or in fact together with the -83 score. From the POV of an analysis user, this is indeed very annoying behavior.
I disagree. If the user wants to see inferior moves and scores, he should use multipv n>1.
User avatar
hgm
Posts: 28265
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: annoying behaviour of stockfish in analysis

Post by hgm »

RubiChess wrote: Sun Nov 10, 2024 7:38 am
hgm wrote: Sat Nov 09, 2024 7:40 pm But you could have printed it before discarding it, or in fact together with the -83 score. From the POV of an analysis user, this is indeed very annoying behavior.
I disagree. If the user wants to see inferior moves and scores, he should use multipv n>1.
Multi-PV is a different engine mode, which causes a serious slowdown. Information about why a previous PV gets refuted at larger depth can be given completely for free. Your remark falls in the same class as saying to someone who complains about potholes in the highway that this is no problem because he could have taken an airplane to fly to his destination...
User avatar
RubiChess
Posts: 639
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: annoying behaviour of stockfish in analysis

Post by RubiChess »

hgm wrote: Sun Nov 10, 2024 10:36 am Information about why a previous PV gets refuted at larger depth can be given completely for free. Your remark falls in the same class as saying to someone who complains about potholes in the highway that this is no problem because he could have taken an airplane to fly to his destination...
The information is implicit there. pv changes -> old pv score was too low. If you fail to get this, you have to take the plane.

In the example above, the additional output looks like this:

info depth 4 seldepth 5 multipv 1 score cp -22 nodes 525 nps 30882 hashfull 0 tbhits 0 time 17 pv e2e6 c6e6 h6e6 d7e6
info depth 5 seldepth 5 multipv 1 score cp -22 upperbound nodes 715 nps 32500 hashfull 0 tbhits 0 time 22 pv e2e6 c6e6
info depth 5 seldepth 6 multipv 1 score cp -22 nodes 734 nps 33363 hashfull 0 tbhits 0 time 22 pv h6e6 c6e6 e2e6 d7e6 e1e2

The old pv just fails low. No information about the exact score of the move but an additional line spamming the UCI log.
User avatar
hgm
Posts: 28265
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: annoying behaviour of stockfish in analysis

Post by hgm »

You get to know the upper bound for the score, and you get to know the move that was found to suppress the score to below that level. None of which follows from the fact that the PV move was changed in that iteration. If you don't understand that, you'd better take the plane...

What you call "spamming the UCI log' is in fact conveying extra information that many people would find useful. Not the people who are only interested in counting the number of wins in an engine match, of corse. To those every info line is spamming of the UCI log; only the bestmove commands are relevant.
Viz
Posts: 223
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: annoying behaviour of stockfish in analysis

Post by Viz »

This only happens because stockfish devs are incompetent people, you might use some much stronger stockfish derivatives like shashchess to get proper analysis that are also much better at higher time controls.