How does Stockfish know it's not the best move, if it hasn't given it a score yet? This is SinglePV behavior, and it's being used in MultiPV.abulmo2 wrote:Not sure to understand. Is that what you want: when a new iteration is started, always display the score of the previous best move, even if it is no more the best move, or in case of multiPV search, one of the k-best moves ?
MutliPV behavior?
Moderators: hgm, Rebel, chrisw
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: MutliPV behavior?
-
- Posts: 433
- Joined: Fri Dec 16, 2016 11:04 am
- Location: France
- Full name: Richard Delorme
Re: MutliPV behavior?
Stockfish gives it a score, but it does not know if it is the best move until all the other moves have been searched. And if a better move is found, it will be displayed in place of the previous best move, so discarding the information you are looking for.Ovyron wrote:How does Stockfish know it's not the best move, if it hasn't given it a score yet? This is SinglePV behavior, and it's being used in MultiPV.abulmo2 wrote:Not sure to understand. Is that what you want: when a new iteration is started, always display the score of the previous best move, even if it is no more the best move, or in case of multiPV search, one of the k-best moves ?
Richard Delorme
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
Re: MutliPV behavior?
Stockfish use aspiration Windows in multi pv. So to find a new bestmove in case of fail low it has to look all the moves, just like in normal search. As soon it has found the new move It start searching for the next move.Ovyron wrote: IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS
Please, try it in some position and tell me you see differently.
In this sense it resolve every fail low.
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: MutliPV behavior?
Yes, thanks for confirming I wasn't going insane with the behavior I was depicting.elcabesa wrote:Stockfish use aspiration Windows in multi pv. So to find a new bestmove in case of fail low it has to look all the moves, just like in normal search.
My claim is that this is indeed best for Stockfish to find a new best move ASAP, give it a score, and show it to the user.
However, for analysis, it makes no sense, because the user also wants to know the score of this best move from previous iteration (and, for all the moves that topped the previous iteration, for that matter), before Stockfish looks at all the moves.
This is also bad when the best move of the previous iteration disappears completely from analysis, because Stockfish finds better moves that equal the number of PV lines, and I have even seen cases where Stockfish misses the best move in such a way, even though it was best, because it never resolves the fail low, but the move fails high after the user makes it on the board and analyzes it.
-
- Posts: 855
- Joined: Sun May 23, 2010 1:32 pm
Re: MutliPV behavior?
since Stockfish use aspiration window to speedup the search the fail low behaviour can't be resolved ( and it's not a problem, it's standard behaviour)
the fact that some move aren't seen until you play them on the board is due to pruning and other strategies used by engines. with all those pruning they are generally stronger, but in some positions they are short-sighted.
the fact that some move aren't seen until you play them on the board is due to pruning and other strategies used by engines. with all those pruning they are generally stronger, but in some positions they are short-sighted.
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: MutliPV behavior?
Yes, this is done intentionally.Ovyron wrote:Did you try the steps I posted? What did you see?elcabesa wrote:give us a reproducible example, with reproducible step and maybe some screenshot of what you thins is worng.
It's useles for me to post reproducible example with reproducible step if people aren't even trying...
IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS
Please, try it in some position and tell me you see differently.
When failing high or low in multipv mode, it doesn't print a new pv.
If you can compile yourself change the following code in search.cpp
Code: Select all
// When failing high/low give some update (without cluttering
// the UI) before a re-search.
if ( mainThread
&& multiPV == 1
&& (bestValue <= alpha || bestValue >= beta)
&& Time.elapsed() > 3000)
sync_cout << UCI::pv(rootPos, rootDepth, alpha, beta) << sync_endl;
Code: Select all
// When failing high/low give some update (without cluttering
// the UI) before a re-search.
if ( mainThread
// && multiPV == 1
&& (bestValue <= alpha || bestValue >= beta)
&& Time.elapsed() > 3000)
sync_cout << UCI::pv(rootPos, rootDepth, alpha, beta) << sync_endl;
Of course, like mentioned in the comment, this may clutter the output in your UI ...
Jörg Oster
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: MutliPV behavior?
Please stop this non-sense and leave programmers forum to ppl that can at least read the code.Ovyron wrote:Did you try the steps I posted? What did you see?
It's useles for me to post reproducible example with reproducible step if people aren't even trying...
IN MULTIPV STOCKFISH IS CHECKING ALTERNATIVE MOVES BEFORE GIVING A SCORE TO THE MAIN MOVE IN FAIL LOWS
Please, try it in some position and tell me you see differently.
It is pretty obvious from SF's code that every search is resolved for each move before moving to the next.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: MutliPV behavior?
Stockfish does resolve fail lows and fail highs, but it resolves them for the root position rather than the (currently) best root move. So at ply 0, not at ply 1.Ovyron wrote:No, it doesn't, you can check on the part of the GUI that shows what move is being analysed, like this:elcabesa wrote: stockfish resolve the fail low/high before moving to any other move when analizyng in multiPV mode.
#/*
Where * is the total number of moves, and # is the number of the move being analyzed.
For a position with 30 available moves, when Stockfish is starting the iteration, it reads:
1/30
Now, pay attention, and you may see the 1 quickly cycling through 2-30. If you stop the engine at that time, Stockfish will show the fail low. If any of the moves 2-30 are better, Stockfish will silently switch to it, and it will appear as new best (i.e. it becomes the 1/30 move.)
Stockfish just hides the fail low on MultiPV, so the user doesn't even know it happened on the output.
All Stockfish derivatives behave like this.
This means that while resolving a fail high or fail low, Stockfish may switch to a new best move and you will not see the final score of the previously best move.
I'm not entirely sure why you are referring to MultiPV here. In MultiPV you do end up with final scores of the best N moves. (I may not have read what you wrote carefully enough though.)
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: MutliPV behavior?
It's because MultiPV is meant to be used for analysis.syzygy wrote:I'm not entirely sure why you are referring to MultiPV here. In MultiPV you do end up with final scores of the best N moves. (I may not have read what you wrote carefully enough though.)
In analysis, the user would have some tree stored somewhere with backsolved scores of positions.
The old behavior allowed the user to insert into their tree the scores of the most interesting moves, even if they exceeded the MultiPV limit. They were given to the user "for free." Now if the user wants to know actual scores of those moves he'd need to make them on the board or rely on go searchmoves to exclude the moves already analyzed from the search.
Here's an example of the old behavior. Let's say that the user analyzes with MultiPV=4 and to depth 28.
At depth 27 (I'm using letters for the moves):
A. 0.29
B. 0.25
C. 0.19
D. 0.10
Now, it starts depth 28.
A has a score of 0.28.
B falls down to a score of 0.07
C falls down to a score of 0.06
D falls down to a score of 0.00
Now, and only now, the engine examines other moves.
E has a score of 0.11
F has a score of 0.09
G has a score of 0.08
The Shredder Classic GUI (up to version 3) has a feature "Continuous Display", that allows the user to see what happened in all the iterations of MultiPV.
The user can add this to the tree:
A. 0.28
E. 0.11
F. 0.09
G. 0.08
B. 0.07
C. 0.06
D. 0.00
That's 7 moves for the price of 4.
With the current behavior:
At depth 27:
A. 0.29
B. 0.25
C. 0.19
D. 0.10
Now, it starts depth 28.
A has a score of 0.28.
B fails low, Stockfish returns E with a score of 0.11
C fails low, Stockfish returns F with a score of 0.09
D fails low, Stockfish returns G with a score of 0.08
The user is left scratching his head, not knowing what happened to B, C, or D (making Stockfish show fail lows doesn't help because those scores aren't safe to add to the tree.) To get those scores, the user would need MultiPV=7, but switching to it now is more costly than old behavior, and the user doesn't know when this is going to happen, so it can't use 7 from the start.
The user can only add this to the tree:
A. 0.28
E. 0.11
F. 0.09
G. 0.08
Now, suppose the user interacts with the positions until A, E, F and G are refuted to a score of <0.08.
In the old bevahior, the user can jump already to B, which is now the best move. With the current behavior, the user has to take means to get a 5th alternative.
What code would need to be changed so that the engine behaves like in Marco Belli's post in Fail Lows?
elcabesa wrote:Stocksfish search the first best result and you'll see the counter going from 1 to 30,
then it search the second best move and you'll see the counter going from 2 to 30,
then it search the third best move and you'll see the counter going from 3 to 30,
and so on ....
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: MutliPV behavior?
But it's actually 7 moves for the price of 7 in terms of search work.
If you want to see the score and PV even of the 7th best move, that is possible by setting MulitiPV to 7. (Of course the 7th move in your list may not be the 7th best move... it might be the 30th best move. Why is it still interesting? Maybe in rare cases it is, but in general?)
If you want to see the score and PV even of the 7th best move, that is possible by setting MulitiPV to 7. (Of course the 7th move in your list may not be the 7th best move... it might be the 30th best move. Why is it still interesting? Maybe in rare cases it is, but in general?)