illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

phenri
Posts: 284
Joined: Tue Aug 13, 2013 9:44 am

illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by phenri »

illogical eval from Stockfish? BBKBPK +- 2.00 then BBKBK = 0.00


[d]8/8/8/4pK2/2k1B3/6B1/8/6b1 w - - 0 1
Stockfish 080114 64 SSE4.2:
(...)
93.Bh4 Bc5 94.Kg4 Kd4 95.Kf3 Kc3 96.Bf6 Kd4 97.Bd8 Bd6 98.Bg5 Kc4 99.Bf6 Kd4 100.Bd8 Bc5 101.Bg5 Kc4 102.Bd2 Kd4 103.Be3+ Kc4 104.Bc1 Kd4 105.Bd2 Kc4 106.Kg4 Kd4 107.Kf5 Kc4 108.Bc1 Bd4 109.Bg5 Bc5 110.Bf6 Kd4 111.Bg5 Kc3 112.Bf6 Kc4 113.Kg4 Kd4 114.Kf5 Ke3 115.Bg5+ Kd4 116.Bd2
+- (2.00) Depth: 46/20 00:00:12 46769kN


Then after Bxe5 its = 0.00
[d]8/8/8/4BK2/2k1B3/8/8/6b1 b - - 0 1

Stockfish 080114 64 SSE4.2:
(...)
93...Bf2 94.Ba1 Be1 95.Bb1 Kd5 96.Ba2+ Kc5 97.Bb1 Bf2 98.Ba2 Be1
= (0.00) Depth: 79/12 00:00:32 111MN
93...Bf2 94.Ba1 Be1 95.Bb1 Kd5 96.Ba2+ Kc5 97.Bb1 Bf2 98.Ba2 Be1
= (0.00) Depth: 80/12 00:00:34 120MN

Why +- 2.00?

nb: eval is without tb's
User avatar
velmarin
Posts: 1600
Joined: Mon Feb 21, 2011 9:48 am

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by velmarin »

I guess the disappearance of the pawn
enters the internal draw some rule.

Same conduct, Komodo TCEC, example.
User avatar
mohzus
Posts: 106
Joined: Tue Sep 24, 2013 2:54 am

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by mohzus »

White has a material advantage of roughly 2 pawns, that's what SF sees. What SF fails to see is that the game is drawn and there's no way to exploit the material advantage. If you play Bxe5, SF will realize it's a draw likely because of the reason mentioned by José.

Analysis of the latest SF with 5 pieces syzygy:

Code: Select all

info depth 53 seldepth 54 score cp 197 nodes 903365490 nps 1344439 tbhits 17198554 time 671927 multipv 1 pv f5g4 g1c5 g4f3 c5b4 g3h4 c4d4 h4f2 d4c4 f3g4 b4d6 f2e1 c4d4 g4f5 d4c4 e1f2 d6c5 f2h4 c5e3 f5g4 c4d4 g4f3 e3d2 h4f6
I wonder what the 6 pieces syzygy analysis can bring.
Anyway I think the position exposes an endgame weakness of SF. There are still many others where SF shows evaluations of +5 in dead draws.
User avatar
velmarin
Posts: 1600
Joined: Mon Feb 21, 2011 9:48 am

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by velmarin »

Bouquet sees an advantage of 0.70 in both cases.

Houdini 4 see the draw in both cases ... (no tables)
but Houdini 3 sees advantage of 3.00 in both cases ....

These are positions where confusion arises, but not going anywhere ...
User avatar
Eelco de Groot
Posts: 4664
Joined: Sun Mar 12, 2006 2:40 am
Full name:   Eelco de Groot

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by Eelco de Groot »

Talking about wrong bishop-endgame evals, here is a real testgame played yesterday where one Stockfish on move 93 promotes to a bishop leading to a same coloured bishop pair. The eval is +76.79. I can't say if here already in high timetrouble but if the annotations are correct I don't think so because up to that point there is only one movetime that was round to 1 second, all other moves rounded to zero seconds so they will not have used up all their time. 100 moves later (no default on time in this one minute per player game. I don't think the engines where playing any random moves even at the end. I saw the game and the moves could still be followed a bit. Probably because of the GUI giving a few miliseconds or so extra? Just my guess) it has to concede only a draw to the other Stockfish :)

Edit: Looking a bit closer, at move 190 it does say {book} just once, so guessing from that it could be a nonsense game in Shredder's book after all and Stockfish is not to blame. That would explain the almost all zero times. But in that case, the other moves should also have been given the {book} comment and I don't think they could have got evals. So that hypothesis is also not satisfactory...

Eelco

[pgn][Event "1 Minutes/Game"]
[Site "Engine Match"]
[Date "2014.01.10"]
[Round "13"]
[White "SfapproximateEval"]
[Black "Stockfish 20131225_003"]
[Result "1/2-1/2"]

1. d4 {book 0s} Nf6 {book 0s} 2. c4 {book 0s} g6 {book 0s}
3. Nc3 {book 0s} Bg7 {book 0s} 4. e4 {book 0s} d6 {book 0s}
5. Nf3 {book 0s} O-O {book 0s} 6. Be2 {book 0s} e5 {book
0s} 7. O-O {book 0s} Nc6 {book 0s} 8. d5 {book 0s} Ne7
{book 0s} 9. Ne1 {book 0s} Nd7 {book 0s} 10. f3 {book 0s}
f5 {book 0s} 11. g4 {book 0s} Kh8 {book 0s} 12. Nd3
{+0.80/13 1s} c6 {-0.90/15 2s} 13. Be3 {+0.82/14 1s} Nb6
{-1.05/14 2s (f4)} 14. b3 {+0.86/15 1s} h5 {-0.96/13 0s
(cxd5)} 15. gxf5 {+1.03/15 1s} gxf5 {-1.35/14 0s} 16. Kh1
{+1.25/15 1s} f4 {-1.45/15 2s (cxd5)} 17. Bf2 {+1.19/15 1s}
Bh3 {-1.59/15 0s (Bf6)} 18. Rg1 {+1.39/16 1s} cxd5
{-1.69/15 1s (Bf6)} 19. exd5 {+1.73/17 5s (cxd5)} Bf5
{-1.61/15 3s (Bd7)} 20. Ne4 {+1.85/15 1s (Rg5)} Ng8
{-1.84/15 3s (Nbc8)} 21. Qf1 {+1.91/15 0s} Rf7 {-1.79/15
1s} 22. Qg2 {+2.32/15 0s} Nh6 {-1.87/15 1s (Bf6)} 23. Ng5
{+2.14/17 1s} Rd7 {-1.95/17 0s} 24. Ne6 {+1.73/17 0s} Qf6
{-1.93/18 3s} 25. Qg5 {+1.72/18 1s} Qxg5 {-2.06/19 2s}
26. Rxg5 {+1.93/19 2s} Bf6 {-2.02/19 0s} 27. Rxh5 {+1.93/19
0s} Rh7 {-2.10/20 0s} 28. Rg1 {+1.91/19 0s} Rg8 {-2.12/19
0s} 29. Ng5 {+1.83/19 0s} Bxg5 {-1.95/19 0s} 30. Rhxg5
{+2.02/19 1s} Rxg5 {-1.83/17 0s} 31. Rxg5 {+1.97/19 0s} Rg7
{-2.06/19 0s} 32. Rg2 {+1.97/20 0s} Nf7 {-2.06/19 0s (Bh3)}
33. a4 {+1.95/17 0s} Nc8 {-2.04/17 0s (Kh7)} 34. a5
{+1.95/17 1s} Kh7 {-2.04/18 2s} 35. Rxg7+ {+1.93/17 1s}
Kxg7 {-2.04/20 0s} 36. Kg2 {+1.93/19 0s} a6 {-1.89/16 1s
(Kf6)} 37. c5 {+1.79/17 1s (b4)} Ne7 {-1.53/16 1s} 38. Nb4
{+1.65/17 0s} dxc5 {-1.73/18 2s} 39. Bxc5 {+1.81/17 1s} Kf6
{-1.65/18 0s (Nc8)} 40. Kf2 {+1.77/18 1s} Nc8 {-1.55/18 0s}
41. h4 {+1.67/18 0s} Ncd6 {-1.81/17 0s (Bb1)} 42. Nd3
{+1.73/14 0s} Nb5 {-2.10/18 1s} 43. Bb6 {+2.00/15 0s} Nbd6
{-2.18/18 1s (Nc3)} 44. Bc7 {+2.10/16 0s} Ke7 {-2.18/18 1s}
45. Bb8 {+2.10/17 0s (Nc5)} Bc8 {-2.40/16 0s (Kf6)} 46. Nc5
{+2.44/18 0s (b4)} Bf5 {-2.34/16 1s} 47. Bd3 {+3.07/17 0s
(h5)} Bxd3 {-2.84/15 0s} 48. Nxd3 {+2.78/17 0s} Kf6
{-2.98/16 0s (Nf5)} 49. h5 {+3.27/17 0s (Bxd6)} Ke7
{-3.55/18 1s (Nf5)} 50. Nc5 {+3.45/18 0s (Ke2)} Ne8
{-3.40/18 0s (Nb5)} 51. Nxb7 {+2.82/19 1s (h6)} Kd7
{-2.76/16 0s} 52. Nc5+ {+2.74/19 0s} Kc8 {-2.86/19 0s}
53. Ba7 {+2.92/21 0s} Nf6 {-2.96/19 0s} 54. Bb6 {+2.97/19
0s} Nxh5 {-2.96/19 0s} 55. Nxa6 {+2.68/19 0s} Nf6 {-2.78/18
0s} 56. Bc5 {+2.94/20 0s} Nxd5 {-2.70/18 0s} 57. Nb4
{+2.64/16 0s} Nc3 {-2.62/15 0s} 58. a6 {+2.36/18 2s} e4
{-2.94/16 0s} 59. Bd4 {+2.62/17 0s} e3+ {-2.88/15 0s}
60. Kf1 {+2.86/17 0s (Ke1)} Nb5 {-2.64/15 0s} 61. Bf6
{+2.80/18 0s} Kb8 {-3.05/15 0s} 62. Nd5 {+2.94/19 0s (Ke2)}
Nfd6 {-3.13/17 0s} 63. Nxf4 {+3.13/17 0s} Nf5 {-3.13/18 0s}
64. Ke2 {+3.09/19 0s} Ka7 {-3.13/18 0s} 65. Nd5 {+3.09/20
0s} Kxa6 {-3.23/20 0s} 66. f4 {+2.98/20 0s} Nbd6 {-3.17/20
0s} 67. Nxe3 {+2.98/21 0s} Kb5 {-3.29/20 0s} 68. Nxf5
{+3.25/22 0s} Nxf5 {-3.29/21 0s} 69. Bc3 {+3.25/21 0s} Nd6
{-3.29/23 0s (Kc5)} 70. Kd3 {+3.25/22 0s} Kc5 {-3.29/23 0s}
71. b4+ {+3.35/22 0s} Kd5 {-3.29/23 0s} 72. Bf6 {+3.43/23
0s (Be5)} Nf5 {-3.29/21 0s (Nb5)} 73. Bd8 {+3.67/21 0s
(Be5)} Nd6 {-3.29/19 0s (Nd4)} 74. Be7 {+3.75/21 0s (Bc7)}
Nf5 {-3.29/23 0s} 75. Bc5 {+4.04/20 0s (Bf6)} Nh4 {-3.61/18
0s (Ng7)} 76. Bf2 {+4.52/19 0s} Ng6 {-3.84/17 0s} 77. Be3
{+4.56/22 0s (f5)} Ne7 {-4.48/15 0s (Nh4)} 78. b5 {+4.56/23
0s} Nf5 {-4.54/17 0s} 79. Bf2 {+4.52/23 0s} Nd6 {-4.66/18
0s} 80. b6 {+4.60/24 0s} Nb7 {-5.46/19 0s} 81. Bd4
{+5.21/20 0s} Kc6 {-5.97/21 0s (Nd8)} 82. Ke4 {+5.69/16 0s
(f5)} Nd8 {-5.97/18 0s} 83. f5 {+5.91/17 0s} Kd6 {-6.14/19
0s} 84. Be5+ {+6.70/17 0s} Ke7 {-6.28/19 0s (Kc6)} 85. Kd5
{+9.83/16 0s} Kd7 {-7.83/19 0s} 86. f6 {+9.91/16 0s} Nb7
{-8.00/19 0s} 87. f7 {+10.14/15 0s (Bc7)} Ke7 {-9.91/12 0s}
88. Kc6 {+23.92/15 0s} Nd8+ {-76.71/13 0s} 89. Kc7
{+76.79/18 0s} Ne6+ {-76.79/17 0s} 90. Kc8 {+76.74/19 0s}
Kxf7 {-76.74/17 0s} 91. b7 {+76.81/18 0s} Kg6 {-76.74/17
0s} 92. Ba1 {+76.79/18 0s (b8B)} Kf5 {-76.81/16 0s}
93. b8=B {+76.79/19 0s} Nc5 {-76.81/16 0s (Nd4)} 94. Kd8
{+76.81/17 0s (Bbe5)} Nb3 {-76.81/15 0s} 95. Bb2 {+76.79/17
0s} Nd4 {-76.84/15 0s (Ke6)} 96. Kd7 {+76.81/18 0s} Nc6
{-76.84/17 0s} 97. Bc7 {+76.84/17 0s} Ne5+ {-76.84/16 0s}
98. Ke7 {+76.84/18 0s} Ke4 {-76.84/16 0s (Nc4)} 99. Ke6
{+76.84/19 0s (Ba1)} Nd3 {-76.84/16 0s} 100. Bh8 {+76.84/20
0s} Nf4+ {-76.84/17 0s} 101. Kd6 {+76.84/21 0s} Nd3
{-76.84/18 0s (Kf3)} 102. Ba1 {+76.84/20 0s (Bb8)} Nf4
{-76.84/17 0s} 103. Bb6 {+76.84/20 0s (Bb8)} Nd3 {-76.84/16
0s (Nd5)} 104. Bbd4 {+76.84/18 0s (Ba7)} Nf4 {-76.84/14 0s
(Ne5)} 105. Ba7 {+76.84/20 0s (Bdc3)} Nd3 {-76.84/16 0s
(Nd5)} 106. Bh8 {+76.84/20 0s (Bb8)} Nf4 {-76.84/15 0s}
107. Bb2 {+76.84/17 0s (Bg7)} Kf3 {-76.84/17 0s} 108. Ba1
{+76.84/20 0s (Bc3)} Ke4 {-76.84/18 0s} 109. Bc3 {+76.84/17
0s (Bb8)} Nd3 {-76.84/17 0s (Nd5)} 110. Bb6 {+76.84/16 0s
(Bb8)} Nf4 {-76.84/16 0s (Ne5)} 111. Bcd4 {+76.84/17 0s
(Bca5)} Nd5 {-76.84/14 0s (Nd3)} 112. Bbc5 {+76.84/19 0s}
Nc3 {-76.84/15 0s (Nf4)} 113. Bh8 {+76.84/19 0s} Nb5+
{-76.84/15 0s (Nd5)} 114. Ke6 {+76.84/19 0s} Nd4+
{-76.84/16 0s} 115. Kf6 {+76.84/19 0s} Nf3 {-76.84/15 0s}
116. Bg7 {+76.84/17 0s (Ba3)} Nd4 {-76.84/13 0s} 117. Bb6
{+76.84/17 0s (Bh6)} Nf3 {-76.84/13 0s (Nf5)} 118. Bh6
{+76.84/16 0s (Ba7)} Ne5 {-76.84/13 0s} 119. Ke6 {+76.84/18
0s} Nd3 {-76.84/13 0s} 120. Ba7 {+76.84/19 0s (Ba5)} Nf4+
{-76.84/12 0s (Ne5)} 121. Ke7 {+76.84/16 0s (Kd6)} Nd5+
{-76.84/11 0s (Kf3)} 122. Kd6 {+76.84/16 0s (Kf7)} Ne7
{-76.84/13 0s (Nf4)} 123. Ke6 {+76.84/16 0s} Nd5 {-76.84/14
0s (Nf5)} 124. Bf8 {+76.84/18 0s (Bb8)} Ne3 {-76.84/13 0s}
125. Bg7 {+76.84/18 0s (Bb8)} Nd5 {-76.84/17 0s} 126. Bh8
{+76.84/18 0s (Bb8)} Nc7+ {-76.84/11 0s (Nf4+)} 127. Kd7
{+76.84/17 0s (Kf6)} Nd5 {-76.84/15 0s} 128. Kd6 {0.00/19
0s (Bb8)} Nf4 {0.00/33 0s} 129. Bhd4 {+76.84/18 0s (Bg7)}
Nd3 {-76.84/14 0s} 130. Bg7 {+76.84/17 0s} Nf4 {-76.84/19
0s} 131. Bb8 {+76.84/17 0s} Nd3 {-76.84/18 0s} 132. Bh8
{+76.84/17 0s (Ba1)} Nf4 {-76.84/19 0s (Ne5)} 133. Ba1
{+76.84/19 0s} Nd3 {-76.84/17 0s (Nd5)} 134. Bg7 {+76.84/16
0s} Nf4 {-76.84/15 0s (Ne5)} 135. Bc3 {+76.84/18 0s (Bh6)}
Nd3 {-76.84/19 0s (Nd5)} 136. Bc7 {+76.84/15 0s (Ba1)} Nf4
{-76.84/12 0s (Ne5)} 137. Be5 {+76.84/15 0s (Bd2)} Kf3
{-76.84/12 0s} 138. Ba1 {0.00/20 0s (Bb6)} Ke4 {0.00/34 0s}
139. Bb2 {+76.71/13 0s (Bb8)} Kf3 {-76.61/8 0s} 140. Bc1
{+12.87/8 0s (Ba3)} Kg2 {-9.30/12 0s (Nd5)} 141. Bxf4
{+9.30/12 0s} Kf2 {-9.30/12 0s (Kf3)} 142. Ke5 {+9.30/10 0s
(Kd5)} Kf3 {-9.30/12 0s} 143. Kf5 {+9.30/11 0s (Bd6)} Ke2
{-9.30/11 0s (Kf2)} 144. Ke4 {+9.30/11 0s} Kf2 {-9.30/12
0s} 145. Bb6+ {+9.30/10 0s (Bb8)} Kg2 {-9.30/11 0s}
146. Be5 {+9.30/10 0s (Bb8)} Kh3 {-9.30/10 0s} 147. Kf3
{+9.30/10 0s} Kh4 {-9.30/11 0s} 148. Kf4 {+9.30/11 0s} Kh5
{-9.35/11 0s (Kh3)} 149. Kf5 {+9.30/10 0s} Kh6 {-9.35/11 0s
(Kh4)} 150. Bc3 {+9.35/10 0s} Kh7 {-9.35/11 0s (Kh5)}
151. Kf6 {+9.35/11 0s} Kh6 {-9.35/11 0s} 152. Bf2 {+9.35/11
0s (Bg1)} Kh5 {-9.35/11 0s} 153. Kf5 {+9.35/12 0s} Kh6
{-9.35/12 0s} 154. Bb2 {+9.35/12 0s (Be3+)} Kh5 {-9.35/12
0s} 155. Bbd4 {+9.35/12 0s (Ba1)} Kh6 {-9.35/12 0s}
156. Bc3 {+9.35/12 0s} Kh5 {-9.35/12 0s} 157. Be5 {+9.35/12
0s (Be3)} Kh6 {-9.35/12 0s} 158. Bed4 {+9.35/12 0s (Bb6)}
Kh5 {-9.35/13 0s} 159. Bg3 {+9.35/12 0s (Ba1)} Kh6
{-9.35/12 0s} 160. Bde5 {+9.35/12 0s (Bge5)} Kh5 {-9.35/12
0s} 161. Bb2 {+9.35/12 0s} Kh6 {-9.35/12 0s} 162. Be1
{+9.35/12 0s (Bf6)} Kh5 {-9.35/11 0s} 163. Bf6 {+9.35/12 0s
(Ba1)} Kh6 {-9.35/12 0s} 164. Bd4 {+9.35/12 0s (Bg3)} Kh5
{-9.35/13 0s} 165. Bg1 {+9.35/12 0s (Ba1)} Kh6 {-9.35/12
0s} 166. Bc3 {+9.35/12 0s} Kh5 {-9.35/13 0s} 167. Bf6
{+9.35/12 0s} Kh6 {-9.35/12 0s} 168. Bh2 {+9.35/13 0s
(Bc5)} Kh7 {-9.30/12 0s (Kh5)} 169. Bc3 {+9.35/12 0s (Bb8)}
Kg8 {-9.35/11 0s (Kh6)} 170. Kf6 {+9.35/11 0s (Bd6)} Kf8
{-9.35/11 0s} 171. Bb2 {+9.35/11 0s (Bg3)} Ke8 {-9.35/11
0s} 172. Ke6 {+9.30/11 0s} Kf8 {-9.35/12 0s (Kd8)} 173. Bg1
{+9.35/11 0s (Bc3)} Ke8 {-9.35/11 0s} 174. Bf6 {+9.35/12
0s} Kf8 {-9.35/12 0s} 175. Bgd4 {+9.35/12 0s (Bc5+)} Ke8
{-9.35/12 0s} 176. Bf2 {+9.35/12 0s (Be3)} Kf8 {-9.35/12
0s} 177. Be1 {+9.35/11 0s (Bg3)} Kg8 {-9.35/11 0s (Ke8)}
178. Bb2 {+9.35/11 0s (Bfc3)} Kf8 {-9.35/12 0s} 179. Bf2
{+9.35/11 0s (Bbc3)} Ke8 {-9.35/11 0s (Kg8)} 180. Bb6
{+9.35/11 0s (Bh4)} Kf8 {-9.35/12 0s} 181. Be3 {+9.35/12
0s} Ke8 {-9.35/12 0s} 182. Bg1 {+9.35/11 0s (Bf6)} Kd8
{-9.30/11 0s} 183. Bb6+ {+9.35/11 0s (Bh2)} Ke8 {-9.35/12
0s} 184. Bc7 {+9.35/12 0s (Bf6)} Kf8 {book 0s} 185. Bg3
{+9.35/12 0s} Kg8 {+M0/1 0s} 186. Bf2 {+9.35/12 0s} Kf8
{+M0/1 0s} 187. Ba7 {+9.35/12 0s} Kg8 {+M0/1 0s (Ke8)}
188. Be3 {+9.10/9 0s} Kf8 {+M0/1 0s (Kh7)} 189. Ba1
{0.00/14 0s} Kg8 {+M0/1 0s} 190. Bc1 {0.00/37 0s} Kf8 {book
0s (Kh7)} 191. Bcb2 {0.00/100 0s} 1/2-1/2[/pgn]

[d]1BK5/8/4n3/5k2/8/8/8/B7 b - -
93. b8=B {+76.79 !}
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
User avatar
mohzus
Posts: 106
Joined: Tue Sep 24, 2013 2:54 am

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by mohzus »

Wow Eelco this is insane!!! I tested it and the position gives +76 with the latest SF as you said!!! This must be close to a world record seriously :D
However the syzygy (5 pieces at least) fix entirely the problem and SF returns +0.00 from depth 1 and onward.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by zullil »

Eelco de Groot wrote:Talking about wrong bishop-endgame evals, here is a real testgame played yesterday where one Stockfish on move 93 promotes to a bishop leading to a same coloured bishop pair. The eval is +76.79.
[d] 2K5/1P6/4n3/5k2/8/8/8/B7 w - - 3 93

This needs fixing. Stockfish doesn't switch to the obvious promotion to a queen until depth 29? :shock: (At least it eventually gets the correct evaluation of mate-in-11.)

Code: Select all

Searching: 2K5/1P6/4n3/5k2/8/8/8/B7 w - - 3 93
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 1  +76.89   00:00      16  b8=B 
 2  +76.72   00:00      48  b8=B Ke4 
 3  +76.82   00:00     121  b8=B Ke4 Kb7 
 4  +76.74   00:00     237  b8=B Ke4 Kb7 Nd4 
 5  +76.74   00:00     362  b8=B Ke4 Kb7 Nd4 Bb2 
 6  +76.74   00:00     535  b8=B Ke4 Kb7 Nd4 Bb2 Nf3 
 7  +76.74   00:00     988  b8=B Ke4 Kb7 Nd8+ Kb6 Nc6 Bg3 Nd4 
 8  +76.74   00:00    2666  b8=B Ke4 Bb2 Nd4 Kd7 Nc6 Bg3 Nd4 
 9  +76.72   00:00    5198  b8=B Ke4 Bb2 Nd4 Kd7 Kf3 Bc3 Ne2 Bh8 
10  +76.74   00:00    9729  b8=B Ke4 Bh8 Nc5 Bh2 Ne6 Kb7 Nd4 Bg3 Nf3 
11  +76.74   00:00   13775  b8=B Ke4 Bh8 Nd4 Kb7 Kf3 Bd6 Ke4 Bh2 Nf3 Bg3 
12  +76.74   00:00   15875  b8=B Ke4 Bh8 Nd4 Kb7 Kf3 Bd6 Ke4 Bh2 Nf3 Bg3 Nd4 
13  +76.74   00:00   45706  b8=B Nc5 Bb2 Ke4 Bh8 Nd3 Kb7 Ne5 Bc7 Kf3 Ba5 Ke4 Be1 
14  +76.84   00:00   69404  b8=B Nc5 Kd8 Ke4 Ke7 Nd3 Kd6 Nf4 Bc7 Nd3 Ba5 Ne5 Ke6 
                            Nd3 
15  +76.84   00:00   87355  b8=B Nc5 Kd8 Ke4 Ke7 Nd3 Kd6 Nf4 Bc7 Nd3 Ba5 Ne5 Ke6 
                            Nd3 Bb6 
16  +76.82   00:00  132155  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Bh2 Ke4 Kc7 Nc4 Kd7 Ne5+ 
                            Ke6 Ng4 Bc7 Kf3 
17  +76.82   00:00  162026  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Bh2 Ke4 Kc7 Nc4 Kd7 Ne3 Bb8 
                            Nc2 Bb2 Ne3 Ke6 Kf3 Bh8 Kg2 
18  +76.82   00:00  204554  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Bh2 Ke4 Kc7 Nc4 Kd7 Ne3 Bb8 
                            Nc2 Bb2 Ne3 Ke6 Kf3 Bh8 Kg2 
19  +76.84   00:00  343335  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Ke8 Nd5 Bb8 Ke4 Kf7 Nf4 Kf6 
                            Nd3 Ke6 Ne5 Ba7 Nd3 Kd6 Ne5 Ke6 
20  +76.84   00:00  481409  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Ke8 Nd5 Bb8 Ke4 Kf7 Nf4 Kf6 
                            Nd3 Ke6 Ne5 Ba7 Nd3 Kd6 Ne5 Bb8 Nf7+ Ke6 
21  +76.84   00:00  745326  b8=B Nc5 Kd8 Nd7 Bc7 Nb6 Ke8 Nd5 Bb8 Ke4 Kf7 Nf4 Kf6 
                            Nd3 Ke6 Ne5 Ba7 Nd3 Kd6 Nf4 Bb8 Nd3 Bc7 Ne5 Ke6 Nd3 
                            Bb8 
22  +76.82   00:00   1561K  b8=B Nc5 Kd8 Ke6 Bh2 Nb3 Bb2 Nc5 Bg1 Nd3 Ba1 Ne5 Ke8 
                            Kf5 Ke7 Nc6+ Kd7 Nd4 Bh2 Ke4 Bb2 Kf3 Bc7 Ke4 Bb6 Kf3 
                            Kd6 Nf5+ Kc6 Ne3 Ba1 Kg2 Bh8 Kf3 Bc7 Kg2 Bb6 
23  +76.82   00:00   1640K  b8=B Nc5 Kd8 Ke6 Bh2 Nb3 Bb2 Nc5 Bg1 Nd3 Ba1 Ne5 Ke8 
                            Kf5 Ke7 Nc6+ Kd7 Nd4 Bh2 Ke4 Bb2 Kf3 Bc7 Ke4 Bb6 Kf3 
                            Kd6 Nf5+ Kc6 Ne3 Ba1 Kg2 Bh8 Kf3 Bc7 Kg2 Bb6 
24  +76.82   00:01   1929K  b8=B Nc5 Kd8 Ke6 Bh2 Nd7 Bc7 Nc5 Ke8 Nb3 Bb2 Nc5 Bh2 
                            Nd3 Ba1 Ne5 Bg1 Nf3 Bb6 Nd4 Bb2 Nf5 Ba7 Nd6+ Kd8 Nc4 
                            Bh8 Ne5 Kc7 Kf5 Bg7 Ke6 Bb8 Kd5 
25  +76.82   00:01   2121K  b8=B Nc5 Kd8 Ke6 Bh2 Nd7 Bc7 Nc5 Ke8 Nb3 Bb2 Nc5 Bh2 
                            Nd3 Ba1 Ne5 Bg1 Nf3 Bb6 Nd4 Bb2 Nf5 Kd8 Nd4 Bc7 Nb5 
                            Bb8 Nd6 Ba1 Nf7+ Kc8 Ne5 Bc7 Kf5 Kd8 Kg4 Ke7 Kh3 Ke6 
                            Kh2 Kd5 
26  +76.82   00:01   2541K  b8=B Nc5 Kd8 Ke6 Bh2 Nd7 Bc7 Nc5 Ke8 Nb3 Bb2 Nc5 Bh2 
                            Nd3 Ba1 Ne5 Bg1 Nf3 Bb6 Nd4 Bb2 Nf5 Kd8 Ne3 Ba5 Kf5 
27  +76.84   00:01   3189K  b8=B Nc5 Kd8 Ke6 Bh2 Nd7 Bc7 Nc5 Ke8 Nb3 Bb2 Nc5 Bh2 
                            Nd3 Ba1 Ne5 Bg3 Kf5 Ke7 Ke4 Kf6 Kf3 Be1 Kg2 Kf5 Nf3 
                            Ba5 Nd4+ Ke4 Nb3 
28  +76.82   00:01   3899K  b8=B Nc5 Kd8 Ke6 Bh2 Nd7 Bc7 Nc5 Ke8 Nb3 Bb2 Nc5 Bh2 
                            Nd7 Ba1 Nf6+ Kf8 Nd7+ Kg7 Ne5 Bg1 Kf5 Bgd4 Nf3 Bdc3 
                            Ne5 Bd4 
29     #11   00:25 290513K  b8=Q Nf4 Qe5+ Kg4 Qe4 Kg5 Bf6+ Kxf6 Qxf4+ Ke6 Kc7 
                            Kd5 Qe3 Kc4 Kc6 Kb4 Qd3 Ka4 Kc5 Ka5 Qb5# 
30     #11   00:29 318053K  b8=Q Nf4 Qe5+ Kg4 Qe4 Kg5 Bf6+ Kxf6 Qxf4+ Ke6 Kc7 
                            Kd5 Qe3 Kc4 Kc6 Kb4 Qd3 Ka4 Kc5 Ka5 Qb5# 
31     #11   00:33 352238K  b8=Q Nf4 Qe5+ Kg4 Qe4 Kg5 Bf6+ Kxf6 Qxf4+ Ke6 Kc7 
                            Kd5 Qe3 Kc4 Kc6 Kb4 Qd3 Ka4 Kc5 Ka5 Qb5# 
32     #11   00:39 395492K  b8=Q Nf4 Qe5+ Kg4 Qe4 Kg5 Bf6+ Kxf6 Qxf4+ Ke6 Kc7 
                            Kd5 Qe3 Kc4 Kc6 Kb4 Qd3 Ka4 Kc5 Ka5 Qb5# 
33     #11   00:43 419812K  b8=Q Nf4 Qe5+ Kg4 Qe4 Kg5 Bf6+ Kxf6 Qxf4+ Ke6 Kc7 
                            Kd5 Qe3 Kc4 Kc6 Kb4 Qd3 Ka4 Kc5 Ka5 Qb5# 
Nodes: 419812094
Nodes/second: 9733193
Best move: b8=Q
Ponder move: Nf4
User avatar
Eelco de Groot
Posts: 4664
Joined: Sun Mar 12, 2006 2:40 am
Full name:   Eelco de Groot

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by Eelco de Groot »

mohzus wrote:Wow Eelco this is insane!!! I tested it and the position gives +76 with the latest SF as you said!!! This must be close to a world record seriously :D
However the syzygy (5 pieces at least) fix entirely the problem and SF returns +0.00 from depth 1 and onward.
It is almost funny is it not :) But Stockfish evaluation is not completely insane, he just does not know that he does not have a real bishop pair...

With a real bishop pair, the "known win" eval would have been right, because the knight can be captured by the the stronger side. Shredderchess tablebases online integrated in the GUI, if you don't have Syzygy, give proof:

[d]1BK5/8/4n3/5k2/8/8/8/1B6 b - -
1... Kg4 -M62
1... Kf6 -M61
1...Kg5 -M60
[www.shredderchess.com]
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
syzygy
Posts: 5697
Joined: Tue Feb 28, 2012 11:56 pm

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by syzygy »

Eelco de Groot wrote:
mohzus wrote:Wow Eelco this is insane!!! I tested it and the position gives +76 with the latest SF as you said!!! This must be close to a world record seriously :D
However the syzygy (5 pieces at least) fix entirely the problem and SF returns +0.00 from depth 1 and onward.
It is almost funny is it not :) But Stockfish evaluation is not completely insane, he just does not know that he does not have a real bishop pair...
Still a real bishop pair should not score higher than Q+B.
With a real bishop pair, the "known win" eval would have been right, because the knight can be captured by the the stronger side.
Quite a few KBBvKN positions need more than 50 moves for capturing the knight. It is not a good idea to score KBBvKN as +75 even if the code made sure it had a real bishop pair.

I'd say special KBBvKN code may be useful:
- to verify that the better side has a real bishop pair, otherwise score as draw;
- if possible to check whether the position can be easily won (I don't know if there are good heuristics for this) and, if so, give a bonus.

But giving a bonus to KBBvKN without giving a bigger bonus to KQBvKN is a bit silly, so K + Q (+ more) versus K + minor should also get a bonus.

Of course this will never give Elo. This will be triggered maybe 10 times in 10000 games. 10 half points in 10000 games is not measurable. The improvement is still real, though. Sometimes common sense prevails over blindly following numbers.
User avatar
mohzus
Posts: 106
Joined: Tue Sep 24, 2013 2:54 am

Re: illogical eval from SF? BBKBPK +- 2.00 then BBKBK = 0.00

Post by mohzus »

Code: Select all

position fen 1BK5/8/4n3/5k2/8/8/8/1B6 b - -
isready
readyok
go infinite
info depth 1 seldepth 1 score cp -15100 nodes 11 nps 141 tbhits 3 time 78 multipv 1 pv f5g4
So SF finds the mate at depth 1, as expected I guess.
At depth 45, there are still 3 tbhits only. Here's the given line:

Code: Select all

info depth 45 seldepth 57 score cp -15100 nodes 38249295 nps 3136988 tbhits 3 time 12193 multipv 1 pv f5g4 c8d7 e6g5 d7e7 g4f3 e7f6 g5e4 f6f5 e4d2 b1c2 d2c4 c2e4 f3e2 f5f4 c4b2 e4c6 b2d3 f4e4 d3f2 e4d4 f2g4 c6e8 g4f6 e8b5 e2f3 b5a4 f3g4 d4e5 g4g5 a4c6 f6g4 e5d5 g5h4 d5e6 h4h3 c6e4 g4h2 e6e5 h2g4 e5f4 g4h2 f4f5 h2g4 e4d3 g4h2 f5e5 h3g2 d3e4 g2h3 e5e6 h2f1 e4c2 f1g3 e6e5 g3f1 c2d1 f1g3 d1a4 h3g4 b8a7
info nodes 38249295 tbhits 3 time 12193
where white's first move is Kd7.