SF not that good at short TC, Komodo very strong

[xr_a_y]

Yesterday I did a Minic vs SF test at suddendeath 10s+0.25 and was surprised by MInic performance, only -150Elo.
In fact this is probably a SF counter performance because of TC management. SF is consuming many time for the middle game and only live on increment in the end-game. So there was two type of games, one were SF find an attacking line and crushed Minic soon, other were SF had to struggle to hold end-game.

I decided to run another tourney with more stronger engines at 10s+0.25 and here is the results

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 komodo-10-linux               246      12    3372   80.5%   23.5%
   2 Defenchess_2.2                101      10    3371   64.2%   33.1%
   3 stockfish                      -3      10    3371   49.6%   27.5%
   4 igel-last                    -130      10    3372   32.2%   30.2%
   5 minic_1.44                   -204      11    3372   23.6%   26.8%

As Komodo succeed to keep its rank and Elo gap at this TC, I wonder if this is not an error in SF short TC management ?

[Raphexon]

SF10-11 has had a few patches that are ELO negative on short TC but ELO positive at longer TC.

Although SF shouldn't be weaker than defencechess.

[xr_a_y]

Look at time management, here for example

Code: Select all

[Event "My Tournament"]
[Site "?"]
[Date "2020.02.22"]
[Round "10"]
[White "minic_1.44"]
[Black "stockfish"]
[Result "1/2-1/2"]
[ECO "A01"]
[GameDuration "00:01:05"]
[GameEndTime "2020-02-22T09:43:57.396 CET"]
[GameStartTime "2020-02-22T09:42:52.276 CET"]
[Opening "Nimzovich-Larsen attack"]
[PlyCount "152"]
[TimeControl "20+0.25"]
[Variation "Classical Variation"]

1. b3 {book} d5 {book} 2. Nf3 {book} Nf6 {book} 3. Bb2 {book} e6 {book}
4. c4 {book} Be7 {book} 5. g3 {book} O-O {book} 6. Bg2 {book} c5 {book}
7. Ne5 {book} Nc6 {book} 8. f4 {book} Nd7 {book} 9. Nxc6 {book} bxc6 {book}
10. Qc2 {book} Rb8 {book} 11. O-O {book} Ba6 {book} 12. d3 {book} Bf6 {book}
13. Bxf6 {+0.41/17 1.2s} Nxf6 {-0.38/16 1.2s} 14. cxd5 {+0.04/17 0.93s}
c4 {-0.66/20 4.6s} 15. bxc4 {+0.10/16 0.76s} exd5 {-0.55/18 1.6s}
16. Nd2 {+0.17/17 1.0s} Ng4 {-0.69/19 3.2s} 17. Rfe1 {-0.19/16 0.91s}
Qf6 {-0.23/18 2.1s} 18. Nf3 {+0.06/15 0.59s} dxc4 {-0.17/18 2.0s}
19. dxc4 {+0.49/14 0.66s} Rb4 {+0.14/17 1.00s} 20. Ng5 {+0.68/15 0.63s}
g6 {-0.57/17 1.1s} 21. e3 {+0.62/15 0.89s} Rb2 {-0.35/17 1.5s}
22. Qd1 {+0.55/16 0.85s} h5 {0.00/19 1.3s} 23. Rb1 {+0.82/15 0.52s}
Rd8 {0.00/16 0.21s} 24. Ne4 {+0.47/18 0.80s} Qe7 {+0.33/20 0.87s}
25. Qxd8+ {-0.01/15 0.43s} Qxd8 {+0.24/17 0.73s} 26. Rxb2 {+0.01/19 0.76s}
Qa5 {+0.34/19 1.2s} 27. Rbb1 {+0.01/19 0.55s} Bxc4 {+0.60/17 0.50s}
28. Rbc1 {-0.01/18 0.71s} Qxa2 {+0.42/16 0.52s} 29. h3 {-0.21/18 0.69s}
Nh6 {+0.60/16 0.30s} 30. Ra1 {-0.46/14 0.41s} Qc2 {+0.76/16 0.26s}
31. Rac1 {-0.33/14 0.46s} Qa2 {+0.79/15 0.25s} 32. Ra1 {-0.37/14 0.36s}
Qb2 {+0.77/15 0.25s} 33. Reb1 {-0.40/15 0.63s} Qc2 {+0.76/16 0.25s}
34. Rc1 {-0.46/16 0.53s} Qd3 {+0.83/17 0.25s} 35. Nf2 {-0.39/14 0.37s}
Qe2 {+0.59/16 0.25s} 36. Re1 {-0.39/14 0.58s} Qb2 {+0.64/14 0.19s}
37. Rab1 {-0.61/13 0.34s} Qf6 {+0.83/15 0.31s} 38. Ne4 {-0.59/13 0.35s}
Qe6 {+1.11/14 0.24s} 39. Rb7 {-0.36/15 0.55s} a5 {+1.02/14 0.26s}
40. Ra7 {-0.63/13 0.38s} Bd5 {+1.04/16 0.25s} 41. Ng5 {-0.74/14 0.30s}
Qd6 {+1.09/15 0.25s} 42. Bxd5 {-0.64/14 0.52s} Qxd5 {+0.80/15 0.25s}
43. Rb1 {-0.79/14 0.51s} Qd2 {+0.91/14 0.25s} 44. Rb8+ {-0.89/16 0.34s}
Kg7 {+2.18/17 0.20s} 45. Re8 {-0.44/16 0.49s} Qc1+ {+0.49/16 0.30s}
46. Kf2 {-0.19/17 0.39s} Qd2+ {+0.31/18 0.25s} 47. Kf1 {-0.12/17 0.29s}
Qd3+ {0.00/18 0.25s} 48. Kf2 {-0.12/18 0.46s} a4 {0.00/19 0.080s}
49. Ne6+ {-0.01/20 0.26s} Kh7 {0.00/21 0.21s} 50. Ng5+ {-0.01/21 0.28s}
Kg7 {+0.58/1 0s} 51. Ne6+ {-0.01/21 0.28s} Kf6 {+0.66/21 0.34s}
52. Ng5 {-0.01/21 0.29s} Qd2+ {+0.32/21 0.55s} 53. Kf1 {-0.01/20 0.28s}
Qd3+ {+0.85/18 0.15s} 54. Kf2 {-0.01/20 0.25s} Qd2+ {+0.55/21 0.42s}
55. Kf1 {-0.01/20 0.26s} Qc1+ {+0.68/19 0.25s} 56. Ke2 {-0.01/20 0.42s}
Qb2+ {+0.27/19 0.25s} 57. Kf3 {-0.01/18 0.43s} Qb1 {0.00/19 0.12s}
58. Kf2 {-0.09/19 0.42s} Qb2+ {0.00/21 0.14s} 59. Kf3 {-0.01/19 0.24s}
Qb6 {0.00/21 0.11s} 60. Ree7 {-0.01/16 0.41s} Qb3 {0.00/17 0.35s}
61. Re8 {-0.01/19 0.24s} Kg7 {0.00/23 0.36s} 62. Ne6+ {-0.01/22 0.33s}
Kf6 {0.00/18 0.37s} 63. Ng5 {-0.01/18 0.36s} Qd5+ {0.00/22 0.098s}
64. Kf2 {-0.01/19 0.26s} Qd3 {0.00/22 0.12s} 65. Rh8 {-0.01/19 0.23s}
h4 {0.00/22 0.26s} 66. Rxh6 {-0.01/20 0.32s} hxg3+ {0.00/22 0.52s}
67. Kf3 {+0.01/25 0.27s} Qd1+ {0.00/16 0.28s} 68. Kxg3 {+0.01/27 0.26s}
Qg1+ {0.00/25 0.27s} 69. Kf3 {-0.01/30 0.26s} Qf1+ {0.00/25 0.25s}
70. Kg3 {+0.01/29 0.27s} Qe1+ {0.00/36 0.25s} 71. Kh2 {-0.01/26 0.22s}
Qf2+ {0.00/34 0.067s} 72. Kh1 {-0.01/31 0.21s} Qf1+ {0.00/21 0.34s}
73. Kh2 {-0.01/32 0.26s} Qe2+ {0.00/28 0.34s} 74. Kh1 {+0.01/36 0.30s}
Qf1+ {0.00/20 0.25s} 75. Kh2 {+0.01/34 0.22s} Qe2+ {0.00/27 0.25s}
76. Kh1 {+0.01/34 0.39s} Qf1+ {0.00/19 0.25s, Draw by 3-fold repetition} 1/2-1/2

4 sec at move 14 ! 3sec at move 16

[Alayan]

I'm skeptical. Bullet TC is typical of fishtest conditions, such a massive regression would never have went unnoticed.

I ran games with asymmetric TC between SF and Ethereal, and SF with 20s+0.2s was demolishing Ethereal with 50+0.5s.

SF with 60s+0.6s tops the FastGM bullet list.

For me the most likely explanation is that something is wrong in how you compiled or set up SF.

[xr_a_y]

I didn't touch default parameters, maybe something are not well suited to this TC.
What Move overhead shall I use, what "slow mover"
And why the hell my UCI_LimitStrenght was activated by default ...

Ok error is on my side probably. Let me try again !

[Alayan]

What is the version you used ?

[xr_a_y]

git master

But maybe I played with my cutechess params some time ago and didn't remember, I reset UCI config.

[xr_a_y]

Ok my bad: the strenght limit was activated. and in fact SF is crazy strong at short TC !!!

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 stockfish                     295      41     285   84.6%   24.6%
   2 komodo-10-linux                91      36     284   62.9%   25.0%
   3 Defenchess_2.2                -24      35     285   46.5%   24.9%
   4 minic_1.44                   -469      69     286    6.3%    8.4%

570 of 6000 games finished.

[Alayan]

That looks better. When getting strange results, most of the time it's because of an unnoticed mistake.