Trusting chess engines

[Uri Blass]

Look at this game (thanks to Ferenc Kiss for showing it):

[Event "Valahol"]
[Site "?"]
[Date "1976.03.02"]
[Round "?"]
[White "Spieler1"]
[Black "Spieler2"]
[Result "0-1"]
[ECO "A49"]
[PlyCount "88"]
[EventType "simul"]
[EventRounds "8"]

1. Nf3 g6 2. g3 Bg7 3. Bg2 Nf6 4. O-O d6 5. d4 O-O 6. c4 Re8 7. Nc3 Bg4 8. Nd2
c6 9. Nde4 Nxe4 10. Nxe4 Qb6 11. c5 dxc5 12. Nxc5 Na6 13. Nb3 Rad8 14. Be3 e5
15. d5 Qb5 16. Re1 cxd5 17. Rc1 b6 18. f3 Be6 19. Bg5 f6 20. Bd2 Nc5 21. e4
Nxb3 22. axb3 dxe4 23. fxe4 Qxb3 24. Qxb3 Bxb3 25. Be3 Bf8 26. Ra1 Rd7 27. Bf1
Red8 28. Be2 Bc5 29. Bxc5 bxc5 30. Rf1 Kg7 31. g4 Ra8 32. h4 a5 33. Rf3 c4 34.
h5 Rd2 35. Kf2 g5 36. Rc3 a4 37. Ke3 Rxb2 38. Bxc4 Bxc4 39. Rxc4 a3 40. Rcc1 a2
41. Kd3 Rg2 42. Rc2 Rg3+ (42... Rc8) 43. Ke2 Rg2+ 44. Kd1 Rg1+ 0-1

It is clear that white is lost after move 43. However, almost all engines in analysis mode show 44. Kd3 with a score of 0. I could find only two engines, Bright and Shredder, who correctly showed a big score for black. It is clear that after 44. Kd3 the position is a two-fold repetition but why should that matter?

It should matter because most programs evaluate first repetition as a draw.

Other options may make the program weaker because it may waste time by searching irrelevant lines and programmers prefer not to spend time on testing changes that are almost irrelevant to playing strength even if they are implemented correctly.

Note that there are other engines except bright and shredder that can see non draw score

Spieler1 - Spieler2, Valahol 1976
[d]r7/6kp/5p2/4p1pP/4P1P1/8/p1R1K1r1/R7 w - - 0 1

Analysis by Yace 0.99.87:

44.Ke2-e1 Rg2xc2
-+ (-11.43) Depth: 1 00:00:00
44.Ke2-f3 Rg2xc2
-+ (-11.37) Depth: 1 00:00:00
44.Ke2-e3 Rg2xc2
-+ (-11.09) Depth: 1 00:00:00
44.Ke2-d1 Rg2xg4
-+ (-5.40) Depth: 1 00:00:00
44.Ke2-d1 Rg2xg4
-+ (-5.40) Depth: 1 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-d2 Rg1xa1
-+ (-7.93) Depth: 2 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-d2 Rg1xa1
-+ (-7.93) Depth: 2 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-d2 Rg1xa1
-+ (-7.93) Depth: 3 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-d2 Rg1xa1
-+ (-7.93) Depth: 3 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-e2 Rg1xa1 46.Rc2-c7+ Kg7-h6
-+ (-10.75) Depth: 4 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-e2 Rg1xa1 46.Rc2-c7+ Kg7-h6
-+ (-10.75) Depth: 4 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-e2 Rg1xa1 46.Rc2-c7+ Kg7-h6 47.Rc7-f7
-+ (-10.74) Depth: 5 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-e2 Rg1xa1 46.Rc2-c7+ Kg7-h6 47.Rc7-f7
-+ (-10.74) Depth: 5 00:00:00
44.Ke2-d1 Rg2-g1+ 45.Kd1-e2 Rg1xa1 46.Rc2-c7+ Kg7-h6 47.Ke2-f2 Ra1-f1+ 48.Kf2xf1 a2-a1Q+ 49.Kf1-g2
-+ (-11.59) Depth: 6 00:00:00 6kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-d1 Rg2-f2
-+ (-11.31) Depth: 6 00:00:00 12kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-d1 Rg2-f2
-+ (-11.31) Depth: 6 00:00:00 12kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-f1 Rc2-b2+ 48.Kb3-c3 Rg2-c2+ 49.Kc3-d3
-+ (-11.32) Depth: 7 00:00:00 17kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-f1 Rc2-b2+ 48.Kb3-c3 Rg2-c2+ 49.Kc3-d3
-+ (-11.32) Depth: 7 00:00:00 17kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-e3 Rg2xc2 46.Ra1-f1 h7-h6 47.Rf1-a1 Rc2-g2 48.Ke3-f3
-+ (-11.29) Depth: 8 00:00:01 52kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-e3 Rg2xc2 46.Ra1-f1 h7-h6 47.Rf1-a1 Rc2-g2 48.Ke3-f3
-+ (-11.29) Depth: 8 00:00:01 53kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-f1 Rc2-b2+ 48.Kb3-a3 Rg2-c2 49.Rf1-a1
-+ (-11.45) Depth: 9 00:00:01 113kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Ra1-f1 Rc2-b2+ 48.Kb3-a3 Rg2-c2 49.Rf1-a1
-+ (-11.45) Depth: 9 00:00:01 115kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.h5-h6+ Kg7xh6 49.Ra1-h1+
-+ (-12.15) Depth: 10 00:00:01 254kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.h5-h6+ Kg7xh6 49.Ra1-h1+
-+ (-12.15) Depth: 10 00:00:01 282kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-a3 Rg2xg4 49.Ka3-b3
-+ (-13.46) Depth: 11 00:00:01 481kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-a3 Rg2xg4 49.Ka3-b3
-+ (-13.46) Depth: 11 00:00:01 577kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-c4 Rg2xg4 49.Kc4-d5
-+ (-14.25) Depth: 12 00:00:01 1012kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-c4 Rg2xg4 49.Kc4-d5
-+ (-14.25) Depth: 12 00:00:01 1104kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-c4 Rg2xg4 49.Kc4-d3
-+ (-14.56) Depth: 13 00:00:02 1894kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b4 Rc8xc2 47.Kb4-b3 Rc2-f2 48.Kb3-c4 Rg2xg4 49.Kc4-d3
-+ (-14.56) Depth: 13 00:00:02 1984kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7xh6 48.Ra1-h1+ Kh6-g6 49.Rh1-a1
-+ (-16.57) Depth: 14 00:00:06 4287kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7xh6 48.Ra1-h1+ Kh6-g6 49.Rh1-a1
-+ (-16.57) Depth: 14 00:00:06 4717kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7xh6 48.Ra1-f1 f6-f5 49.Rf1-h1+
-+ (-16.82) Depth: 15 00:00:11 8056kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7xh6 48.Ra1-f1 f6-f5 49.Rf1-h1+
-+ (-16.82) Depth: 15 00:00:11 8435kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Kb3-b4 Rc2-b2+ 48.Kb4-a3 Rb2-e2
-+ (-17.14) Depth: 16 00:00:21 15308kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.Kb3-b4 Rc2-b2+ 48.Kb4-a3 Rb2-e2
-+ (-17.14) Depth: 16 00:00:22 15694kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7-g8 48.Kb3-a3 Rc2-c3+ 49.Ka3-b4
-+ (-25.43) Depth: 17 00:00:50 36665kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7-g8 48.Kb3-a3 Rc2-c3+ 49.Ka3-b4
-+ (-25.43) Depth: 17 00:00:58 43047kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7-g8 48.Kb3-a3 Rc2-c3+ 49.Ka3-b4
-+ (-25.46) Depth: 18 00:01:29 65388kN
44.Ke2-d3 Ra8-d8+ 45.Kd3-c3 Rd8-c8+ 46.Kc3-b3 Rc8xc2 47.h5-h6+ Kg7-g8 48.Kb3-a3 Rc2-c3+ 49.Ka3-b4
-+ (-25.46) Depth: 18 00:01:30 66725kN

(, 02.08.2007)

[Jim Walker]

My question is: Why would any engine play 42....Rg3+ when 42....Rd8 is available and wins quickly?
Jim

[hgm]

Because most engines are not programmed to prefer quick wins over slow wins. They use the minimax algorithm, which assigns scores to the moves only based on the final position the will lead to (against best opponent play), and ignore completely how you get there.

If it were not for the explicit programming of repetition = draw, engines might very well play Rg3+, Rg2+ forever, considering both these moves to win a rook all the time. The only thing that makes them finally decide to take the rook is that sooner or later any alternative to postpone it gets exhausted, as they are all reassigned a draw score (from their original qualification as a rook win) after having occurred in the game (once or twice, depending on the engine).

(See also http://www.talkchess.com/forum/viewtopic.php?t=15496 )

[Jim Walker]

????
You talk a good game but your argument is nonsense. I tried 15 engines and not one of them will miss 42..... Rd8 with a score of +9 or more.
Jim

[Vempele]

????
You talk a good game but your argument is nonsense. I tried 15 engines and not one of them will miss 42..... Rd8 with a score of +9 or more.
Jim

That's because you didn't read his second paragraph. All those engines are explicitly programmed to know that repetition = draw.

[Jim Walker]

Did you read what he said about "most engines"? That is what I was responding to. (In reply to my post) Most engines will not miss the early mate is my simple conclusion.
Jim

[Pradu]

????
You talk a good game but your argument is nonsense. I tried 15 engines and not one of them will miss 42..... Rd8 with a score of +9 or more.
Jim

Don't be rude to poor HG.

r7/6kp/5p2/4p1pP/4P1P1/3K4/p1R3r1/R7 b - - 0 42 (call this position A)
Play moves Rg3+ Ke2 Kd3 Rg3+ (2 fold repetion).

Many engines are programmed such that they decide a 2 fold repetition is a draw in search, because the position after Ke2 is a 2-fold repetition, it is evaluated as a draw.

Why are most engines programmed so that 2-fold repetition is a draw? This is to not waste nodes searching useless nodes in search. Perhaps it would help if you read this paper for a more detailed explanation on this matter:
http://www.seanet.com/~brucemo/topics/repetition.htm

On a side node: Yes, you cannot always trust chess engines because they prune like crazy. But the same reason could also go for humans who prune even more....

[Jim Walker]

I don't think I was rude but if so I appoligize. I thought I was simply being blunt. Most engines would not miss the early mate when playing in a game which is what he was implying. Forcing the moves past the early mate and then backing up or some other such action may cause strange effects which engines would not do under ordinary circumstances. I guess some people do this when analyzing games. I didn't understand that he was analyzing a game and then this "feature" showed up.

Jim

[hgm]

Note that the given position is not about mate, but bout winning a rook. Most engines do implement an ad-hoc solution for preferring fast mates (making the evaluation of a checkmate position dependent on its level in the tree).

That engines do score fast wins equal to slow wins des of course not mean that the will always prefer the slow win. he move choice in such a case will be determined by unintended factors, such as move sorting. Sometimes this helps t prefer the fast win, if the slow one cannot be seen at first, but the fast one can (so that it looks a lot better), and moves are sorted on expected merit in between.

There is no guarantee they will always do that, though. By the time the engine searches deep enough to see both lines win a rook, it might intentionally stall to push something bad that can happen after it beyond the horizon.