MutliPV behavior?

[Ovyron]

Not sure to understand. Is that what you want: when a new iteration is started, always display the score of the previous best move, even if it is no more the best move, or in case of multiPV search, one of the k-best moves ?

How does Stockfish know it's not the best move, if it hasn't given it a score yet? This is SinglePV behavior, and it's being used in MultiPV.

[abulmo2]

Not sure to understand. Is that what you want: when a new iteration is started, always display the score of the previous best move, even if it is no more the best move, or in case of multiPV search, one of the k-best moves ?

How does Stockfish know it's not the best move, if it hasn't given it a score yet? This is SinglePV behavior, and it's being used in MultiPV.

Stockfish gives it a score, but it does not know if it is the best move until all the other moves have been searched. And if a better move is found, it will be displayed in place of the previous best move, so discarding the information you are looking for.

[elcabesa]

IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS

Please, try it in some position and tell me you see differently.

Stockfish use aspiration Windows in multi pv. So to find a new bestmove in case of fail low it has to look all the moves, just like in normal search. As soon it has found the new move It start searching for the next move.
In this sense it resolve every fail low.

[Ovyron]

Stockfish use aspiration Windows in multi pv. So to find a new bestmove in case of fail low it has to look all the moves, just like in normal search.

Yes, thanks for confirming I wasn't going insane with the behavior I was depicting.

My claim is that this is indeed best for Stockfish to find a new best move ASAP, give it a score, and show it to the user.

However, for analysis, it makes no sense, because the user also wants to know the score of this best move from previous iteration (and, for all the moves that topped the previous iteration, for that matter), before Stockfish looks at all the moves.

This is also bad when the best move of the previous iteration disappears completely from analysis, because Stockfish finds better moves that equal the number of PV lines, and I have even seen cases where Stockfish misses the best move in such a way, even though it was best, because it never resolves the fail low, but the move fails high after the user makes it on the board and analyzes it.

[elcabesa]

since Stockfish use aspiration window to speedup the search the fail low behaviour can't be resolved ( and it's not a problem, it's standard behaviour)

the fact that some move aren't seen until you play them on the board is due to pruning and other strategies used by engines. with all those pruning they are generally stronger, but in some positions they are short-sighted.

[Joerg Oster]

give us a reproducible example, with reproducible step and maybe some screenshot of what you thins is worng.

Did you try the steps I posted? What did you see?

It's useles for me to post reproducible example with reproducible step if people aren't even trying...

IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS

Please, try it in some position and tell me you see differently.

Yes, this is done intentionally.
When failing high or low in multipv mode, it doesn't print a new pv.

If you can compile yourself change the following code in search.cpp

Code: Select all

              // When failing high/low give some update (without cluttering
              // the UI) before a re-search.
              if (   mainThread
                  && multiPV == 1
                  && &#40;bestValue <= alpha || bestValue >= beta&#41;
                  && Time.elapsed&#40;) > 3000&#41;
                  sync_cout << UCI&#58;&#58;pv&#40;rootPos, rootDepth, alpha, beta&#41; << sync_endl;

to

Code: Select all

              // When failing high/low give some update (without cluttering
              // the UI) before a re-search.
              if (   mainThread
              //    && multiPV == 1
                  && &#40;bestValue <= alpha || bestValue >= beta&#41;
                  && Time.elapsed&#40;) > 3000&#41;
                  sync_cout << UCI&#58;&#58;pv&#40;rootPos, rootDepth, alpha, beta&#41; << sync_endl;

and you get new output on every fail-low or fail-high.
Of course, like mentioned in the comment, this may clutter the output in your UI ...

[Milos]

Did you try the steps I posted? What did you see?

It's useles for me to post reproducible example with reproducible step if people aren't even trying...

IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS

Please, try it in some position and tell me you see differently.

Please stop this non-sense and leave programmers forum to ppl that can at least read the code.
It is pretty obvious from SF's code that every search is resolved for each move before moving to the next.

[syzygy]

stockfish resolve the fail low/high before moving to any other move when analizyng in multiPV mode.

No, it doesn't, you can check on the part of the GUI that shows what move is being analysed, like this:

#/*

Where * is the total number of moves, and # is the number of the move being analyzed.

For a position with 30 available moves, when Stockfish is starting the iteration, it reads:

1/30

Now, pay attention, and you may see the 1 quickly cycling through 2-30. If you stop the engine at that time, Stockfish will show the fail low. If any of the moves 2-30 are better, Stockfish will silently switch to it, and it will appear as new best (i.e. it becomes the 1/30 move.)

Stockfish just hides the fail low on MultiPV, so the user doesn't even know it happened on the output.

All Stockfish derivatives behave like this.

Stockfish does resolve fail lows and fail highs, but it resolves them for the root position rather than the (currently) best root move. So at ply 0, not at ply 1.

This means that while resolving a fail high or fail low, Stockfish may switch to a new best move and you will not see the final score of the previously best move.

I'm not entirely sure why you are referring to MultiPV here. In MultiPV you do end up with final scores of the best N moves. (I may not have read what you wrote carefully enough though.)

[Ovyron]

I'm not entirely sure why you are referring to MultiPV here. In MultiPV you do end up with final scores of the best N moves. (I may not have read what you wrote carefully enough though.)

It's because MultiPV is meant to be used for analysis.

In analysis, the user would have some tree stored somewhere with backsolved scores of positions.

The old behavior allowed the user to insert into their tree the scores of the most interesting moves, even if they exceeded the MultiPV limit. They were given to the user "for free." Now if the user wants to know actual scores of those moves he'd need to make them on the board or rely on go searchmoves to exclude the moves already analyzed from the search.

Here's an example of the old behavior. Let's say that the user analyzes with MultiPV=4 and to depth 28.

At depth 27 (I'm using letters for the moves):

A. 0.29
B. 0.25
C. 0.19
D. 0.10

Now, it starts depth 28.

A has a score of 0.28.
B falls down to a score of 0.07
C falls down to a score of 0.06
D falls down to a score of 0.00

Now, and only now, the engine examines other moves.

E has a score of 0.11
F has a score of 0.09
G has a score of 0.08

The Shredder Classic GUI (up to version 3) has a feature "Continuous Display", that allows the user to see what happened in all the iterations of MultiPV.

The user can add this to the tree:

A. 0.28
E. 0.11
F. 0.09
G. 0.08
B. 0.07
C. 0.06
D. 0.00

That's 7 moves for the price of 4.

With the current behavior:

At depth 27:

A. 0.29
B. 0.25
C. 0.19
D. 0.10

Now, it starts depth 28.

A has a score of 0.28.
B fails low, Stockfish returns E with a score of 0.11
C fails low, Stockfish returns F with a score of 0.09
D fails low, Stockfish returns G with a score of 0.08

The user is left scratching his head, not knowing what happened to B, C, or D (making Stockfish show fail lows doesn't help because those scores aren't safe to add to the tree.) To get those scores, the user would need MultiPV=7, but switching to it now is more costly than old behavior, and the user doesn't know when this is going to happen, so it can't use 7 from the start.

The user can only add this to the tree:

A. 0.28
E. 0.11
F. 0.09
G. 0.08

Now, suppose the user interacts with the positions until A, E, F and G are refuted to a score of <0.08.

In the old bevahior, the user can jump already to B, which is now the best move. With the current behavior, the user has to take means to get a 5th alternative.

What code would need to be changed so that the engine behaves like in Marco Belli's post in Fail Lows?

Stocksfish search the first best result and you'll see the counter going from 1 to 30,
then it search the second best move and you'll see the counter going from 2 to 30,
then it search the third best move and you'll see the counter going from 3 to 30,

and so on ....

[syzygy]

But it's actually 7 moves for the price of 7 in terms of search work.
If you want to see the score and PV even of the 7th best move, that is possible by setting MulitiPV to 7. (Of course the 7th move in your list may not be the 7th best move... it might be the 30th best move. Why is it still interesting? Maybe in rare cases it is, but in general?)