illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Moderator: Ras
-
- Posts: 28361
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Well, being useful to correspondence players doesn't give you any extra Elo, so it seems to be a non-negociable no-no for Stockfish...
-
- Posts: 5697
- Joined: Tue Feb 28, 2012 11:56 pm
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Well, every engine has to make a choice on how to deal with repetitions.hgm wrote:Call it what you like, but that doesn't make it any less wrong.syzygy wrote:This is intentional, so not a bug.
Some engines only score a repetition as a draw if the position occurs for the 3rd time or if the position occurs for the 2nd time in the search tree. This is completely "correct". This is what my engine does.
Most engines each each repetition as a draw, so also if it occurs only once in the history of the root position and once in the search tree. This has the problem that if the opponent messes up in a won position, the engine might give him a second chance by moving back into this position thinking that it will draw. However, it has also advantages:
- draw by reps are detected earlier;
- the engine will not repeat positions if it thinks it is better.
My engine regularly repeats positions once while thinking it is ahead and only chooses another path the third time. This looks pretty bad, don't you think?
-
- Posts: 5697
- Joined: Tue Feb 28, 2012 11:56 pm
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
SF probably should not display Qf4, since it is assuming a draw after 88.Kd1. Displaying 88...Qf4 might be a bug.phenri wrote:...do not tell me that you find it normal (even to the point of view of a developer or an end user), because as a result of repeated movement, the engine does not suggest the right move?
It has none of these implications.This could have implications if interacting with table bases, and they suggest moves that could be repeated but Stockfish (or GUI, I do not know) refuses to show, it becomes problematic for by example a correspondance player or freestyle)
Regarding correspondence play, it will only happen if the opponent plays a rather bad move in a good position and manages to find a better move on the 2nd try. That could happen yes, but if a correspondence player cannot handle this, I think he should look for a different hobby.
-
- Posts: 284
- Joined: Tue Aug 13, 2013 9:44 am
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
According to the rules of FIDE, 3 repetitions would be needed so that it is drawing, except here there are not 3 repetitions.syzygy wrote:Well, every engine has to make a choice on how to deal with repetitions.hgm wrote:Call it what you like, but that doesn't make it any less wrong.syzygy wrote:This is intentional, so not a bug.
Some engines only score a repetition as a draw if the position occurs for the 3rd time or if the position occurs for the 2nd time in the search tree. This is completely "correct". This is what my engine does.
Most engines each each repetition as a draw, so also if it occurs only once in the history of the root position and once in the search tree. This has the problem that if the opponent messes up in a won position, the engine might give him a second chance by moving back into this position thinking that it will draw. However, it has also advantages:
- draw by reps are detected earlier;
- the engine will not repeat positions if it thinks it is better.
My engine regularly repeats positions once while thinking it is ahead and only chooses another path the third time. This looks pretty bad, don't you think?
If I includes your reasoning, it is for you normal that the engine plays a move which belongs to the worst is possible be losing and at best all that is drawing, to avoid a repetition which is winning and in the worst drawing?
Well, apparently everyone who cares, especially those interested...
-
- Posts: 284
- Joined: Tue Aug 13, 2013 9:44 am
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
PV 88.Kd1 Qf4 is obviously a bug, except that few people recognize.syzygy wrote: SF probably should not display Qf4, since it is assuming a draw after 88.Kd1. Displaying 88...Qf4 might be a bug.
I was referring to cases of complicated positions finals and not easy to win when it is possible that the engine may suggest one or two repeated moves.syzygy wrote:It has none of these implications.This could have implications if interacting with table bases, and they suggest moves that could be repeated but Stockfish (or GUI, I do not know) refuses to show, it becomes problematic for by example a correspondance player or freestyle)
Regarding correspondence play, it will only happen if the opponent plays a rather bad move in a good position and manages to find a better move on the 2nd try. That could happen yes, but if a correspondence player cannot handle this, I think he should look for a different hobby.
-
- Posts: 10825
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
88...Kf4 in the pv is not normal but 88.Kd1 with 0.00 score is clearly normal and this is a behaviour of many engines.phenri wrote:Just to illustrate my point ...
![]()
...do not tell me that you find it normal (even to the point of view of a developer or an end user), because as a result of repeated movement, the engine does not suggest the right move?
This could have implications if interacting with table bases, and they suggest moves that could be repeated but Stockfish (or GUI, I do not know) refuses to show, it becomes problematic for by example a correspondance player or freestyle)
I do not find it to be a problem in analysis because I know for many years that I cannot trust analysis of engines when they show a line that leads to repetition in the pv(assuming one of the previous moves was wrong) and need to give them the FEN without previous moves to get a correct analysis.
-
- Posts: 28361
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Yes, it does. But not as bad as not finding a checkmate.syzygy wrote:My engine regularly repeats positions once while thinking it is ahead and only chooses another path the third time. This looks pretty bad, don't you think?
It should also be possible to prevent this. You could score the first repetition of a game-history position not as a draw, but just scale its score (if it is not a mating score) towards a draw. This should appreciably reduce the chances for needless repetitions.
Another possibility is to make a distinction between moves the engine played, and moves that were forced by the user. The whole logic that the the engine would have found the win first time around if there was one sort of crumbles when the move was forced that first time. And when analyzing, moves are by definition all forced by the user.
-
- Posts: 974
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
- Full name: Jörg Oster
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Nothing wrong with that.syzygy wrote:Well, every engine has to make a choice on how to deal with repetitions.hgm wrote:Call it what you like, but that doesn't make it any less wrong.syzygy wrote:This is intentional, so not a bug.
Some engines only score a repetition as a draw if the position occurs for the 3rd time or if the position occurs for the 2nd time in the search tree. This is completely "correct". This is what my engine does.
Most engines each each repetition as a draw, so also if it occurs only once in the history of the root position and once in the search tree. This has the problem that if the opponent messes up in a won position, the engine might give him a second chance by moving back into this position thinking that it will draw. However, it has also advantages:
- draw by reps are detected earlier;
- the engine will not repeat positions if it thinks it is better.
My engine regularly repeats positions once while thinking it is ahead and only chooses another path the third time. This looks pretty bad, don't you think?
As long as it doesn't go for the 3-fold-rep, it even looks a bit human.
Many human players do this, for time-saving reasons, or for psycohological ones. The opponent thinks, oh great, he goes for the draw. And then, surprise, surprise...

But seriously, as long as the 3-fold-rep is being avoided, I wouldn't care much.
Better than
Jörg Oster
-
- Posts: 5697
- Joined: Tue Feb 28, 2012 11:56 pm
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
In theory the "drawing move" might lead to the engine quickly being checkmated, but that can only happen if the opponent overlooked the checkmate the first time around. In engine-engine matches this is unlikely to happen (and this includes correspondence play). In engine-human matches it could happen but the chances are rather slim.hgm wrote:Yes, it does. But not as bad as not finding a checkmate.syzygy wrote:My engine regularly repeats positions once while thinking it is ahead and only chooses another path the third time. This looks pretty bad, don't you think?
I think the search would simply look for a PV not ending in a position that has been seen before. I doubt it would do much to prevent needless repetitions.It should also be possible to prevent this. You could score the first repetition of a game-history position not as a draw, but just scale its score (if it is not a mating score) towards a draw. This should appreciably reduce the chances for needless repetitions.
More importantly, it seems the choice of how to deal with this issue measurably affects Elo, the solution picked by SF and most other engines simply being the "best". The downside is mostly cosmetic.
A special mode for analysis purposes would of course be possible.Another possibility is to make a distinction between moves the engine played, and moves that were forced by the user. The whole logic that the the engine would have found the win first time around if there was one sort of crumbles when the move was forced that first time. And when analyzing, moves are by definition all forced by the user.
-
- Posts: 5697
- Joined: Tue Feb 28, 2012 11:56 pm
Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00
Sure, but SF does not adjudicate the position as a draw. SF only scores the position as a draw. For any engine you can find zillions of positions for which the engine returns a score that does not conform to the game-theoretic outcome.phenri wrote:According to the rules of FIDE, 3 repetitions would be needed so that it is drawing, except here there are not 3 repetitions.
I find it hard to understand this sentence. In any event, it is normal for any engine to not always play the very best move.If I includes your reasoning, it is for you normal that the engine plays a move which belongs to the worst is possible be losing and at best all that is drawing, to avoid a repetition which is winning and in the worst drawing?
The developers of SF have chosen to implement repetition detection in a way that overall most often results in the best move being played, i.e. the implementation that is measurably best in terms of Elo. The downside is a cosmetic one. I find it difficult to criticise the SF developers for not addressing a mostly cosmetic issue which would cost real Elo to solve.