Poor performance of Stockfish 10 against Igel 1.8.3 at (shallow) fixed depth=10
Posted: Thu Aug 01, 2019 1:33 am
Hello everyone,
I am trying to hunt down some issues in Igel (CCRL 40/40 ~2800 elo) when running games at a very quick time control, and one of my ideas was to measure the "time to depth" in Igel as well as see if I should improve evaluation or search.
So I decided to run a few games between Stockfish 10 and Igel 1.8.3 at a fixed depth=10 and the results are very surprising to me:
How come the performance of Stockfish is ~39 elo worse than Igel at a fixed depth=10? Here is an example of one game:
As you can see, around move 11 the evaluation of stockfish drops a lot:
... yet the time difference spent on a move between two moves is not so big: 0.73s versus 0.015s (a factor of x48), then the score for black suddenly drops and the game is pretty much over.
I can't rationally explain this difference in performance. Perhaps a bug in SF at shallow depths?
I am trying to hunt down some issues in Igel (CCRL 40/40 ~2800 elo) when running games at a very quick time control, and one of my ideas was to measure the "time to depth" in Igel as well as see if I should improve evaluation or search.
So I decided to run a few games between Stockfish 10 and Igel 1.8.3 at a fixed depth=10 and the results are very surprising to me:
Code: Select all
tc=inf
depth=10
Score of Igel 1.8.3 64 POPCNT vs Stockfish 10 64 POPCNT: 105 - 79 - 45 [0.557] 229
Elo difference: 39.62 +/- 40.67
Code: Select all
[Event "?"]
[Site "?"]
[Date "2019.07.31"]
[Round "1"]
[White "Igel 1.8.3 64 POPCNT"]
[Black "Stockfish 10 64 POPCNT"]
[Result "1-0"]
[ECO "B04"]
[Opening "Alekhine's defense"]
[PlyCount "110"]
[Termination "adjudication"]
[TimeControl "inf"]
[Variation "Modern, Fianchetto Variation"]
1. e4 {book} Nf6 {book} 2. e5 {book} Nd5 {book} 3. d4 {book} d6 {book}
4. Nf3 {book} g6 {book} 5. Bc4 {book} Nb6 {book} 6. Bb3 {book} Bg7 {book}
7. Ng5 {book} e6 {book} 8. Qf3 {book} Qe7 {book} 9. Ne4 {+0.15/10 1.1s}
O-O {-0.52/10 0.021s} 10. Nf6+ {+0.70/10 0.34s} Kh8 {-0.05/10 0.004s}
11. Bg5 {+0.66/10 0.73s} h6 {-1.17/10 0.015s} 12. Qh3 {+1.96/10 0.87s}
N8d7 {-0.52/10 0.020s} 13. Ne4 {+1.85/10 0.58s} Qe8 {-3.75/10 0.008s}
14. Bxh6 {+2.67/10 0.66s} Kg8 {-3.15/10 0.004s} 15. exd6 {+2.70/10 0.44s}
Bxd4 {-4.12/10 0.011s} 16. Nbc3 {+3.10/10 0.43s} Nd5 {-4.16/10 0.029s}
17. dxc7 {+3.57/10 1.9s} Nxc3 {-4.98/10 0.015s} 18. Nxc3 {+3.69/10 0.26s}
Bg7 {-5.06/10 0.016s} 19. Bxg7 {+3.96/10 0.98s} Kxg7 {-4.31/10 0.012s}
20. Ne4 {+3.92/10 0.50s} Qe7 {-4.68/10 0.004s} 21. O-O-O {+3.79/10 0.57s}
Nf6 {-4.31/10 0.002s} 22. Qc3 {+3.95/10 0.48s} e5 {-4.55/10 0.006s}
23. Nxf6 {+4.00/10 0.54s} Qxf6 {-5.03/10 0.017s} 24. Rhe1 {+4.00/10 0.003s}
Re8 {-5.00/10 0.022s} 25. Rd5 {+3.88/10 0.69s} e4 {-5.20/10 0.009s}
26. Rd4 {+4.15/10 1.3s} Bf5 {-4.12/10 0.009s} 27. h3 {+3.43/10 1.0s}
Rac8 {-4.27/10 0.011s} 28. g4 {+3.53/10 0.15s} Be6 {-4.23/10 0.007s}
29. Rexe4 {+3.73/10 0.58s} Re7 {-3.43/10 0.007s} 30. Rf4 {+3.41/10 0.89s}
Rexc7 {-3.65/10 0.004s} 31. Rxf6 {+3.33/10 0.23s} Rxc3 {-3.37/10 0.001s}
32. Rxe6 {+3.33/10 0.012s} Rxb3 {-3.36/10 0.010s} 33. axb3 {+3.35/10 0.080s}
fxe6 {-3.67/10 0.004s} 34. Rd7+ {+3.35/10 0.001s} Kf6 {-3.48/10 0.001s}
35. Rxb7 {+3.35/10 0s} Rh8 {-3.21/10 0.002s} 36. Rxa7 {+3.81/10 0.054s}
Rxh3 {-3.20/10 0.004s} 37. b4 {+3.81/10 0s} Rf3 {-3.85/10 0.004s}
38. b5 {+4.56/10 0.038s} Ke5 {-4.31/10 0.007s} 39. b6 {+4.00/10 0.050s}
Kd6 {-3.25/10 0.003s} 40. Rg7 {+4.41/10 0.093s} Rxf2 {-3.16/10 0.003s}
41. b7 {+4.92/10 0.041s} Rf8 {-4.21/10 0.005s} 42. Rxg6 {+5.16/10 0.10s}
Kc6 {-4.27/10 0.004s} 43. Rg7 {+5.96/10 0.11s} Kb6 {-4.99/10 0.008s}
44. g5 {+6.02/10 0.11s} Ka7 {-5.38/10 0.004s} 45. g6 {+6.18/10 0.038s}
Rf6 {-5.61/10 0.008s} 46. Kd2 {+6.23/10 0.058s} e5 {-5.78/10 0.005s}
47. Kd3 {+6.79/10 0.064s} e4+ {-5.39/10 0.005s} 48. Kxe4 {+7.56/10 0.093s}
Rb6 {-6.53/10 0.012s} 49. c4 {+8.25/10 0.051s} Rb4 {-8.14/10 0.008s}
50. Rf7 {+10.17/10 0.046s} Rxc4+ {-6.69/10 0.003s} 51. Kd5 {+14.07/10 0.050s}
Rb4 {-13.06/10 0.005s} 52. g7 {+14.07/10 0.008s} Rb5+ {-55.80/10 0.005s}
53. Kc6 {+21.22/10 0.078s} Rb6+ {-13.49/10 0.002s} 54. Kc7 {+15.66/10 0.11s}
Rxb7+ {-7.22/10 0.005s} 55. Kd8 {+14.59/10 0.17s}
Rxf7 {-6.06/10 0.009s, White wins by adjudication: SyzygyTB} 1-0
Code: Select all
11. Bg5 {+0.66/10 0.73s} h6 {-1.17/10 0.015s}
I can't rationally explain this difference in performance. Perhaps a bug in SF at shallow depths?