HERT book is drawish

[Jouni]

Another match:

Code: Select all

                                         
1   Stockfish 16.1                 +0/=200/-0 50.00%  100.0/200  -6384.00
2   Stockfish dev-20240709-362a77  +0/=200/-0 50.00%  100.0/200  -6384.00

Or chess is solved ?!

[MMarco]

I ran 30sec + 0.5sec on single threaded Stockfish 16.1 against itself with HERT lowdraw 250 positions.

Code: Select all

Score of Stockfish 16.1 vs Stockfish 16.1_copy: 13 - 18 - 469 [0.495]
...      Stockfish 16.1 playing White: 7 - 5 - 238  [0.504] 250
...      Stockfish 16.1 playing Black: 6 - 13 - 231  [0.486] 250
...      White vs Black: 20 - 11 - 469  [0.509] 500
Elo difference: -3.5 +/- 7.6, LOS: 18.5 %, DrawRatio: 93.8 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

94% draws. Using Stockfish 11 against itself the draw rate lowers to 71%

Code: Select all

Score of Stockfish 11 vs Stockfish 11_copy: 75 - 68 - 357 [0.507]
...      Stockfish 11 playing White: 57 - 17 - 176  [0.580] 250
...      Stockfish 11 playing Black: 18 - 51 - 181  [0.434] 250
...      White vs Black: 108 - 35 - 357  [0.573] 500
Elo difference: 4.9 +/- 16.3, LOS: 72.1 %, DrawRatio: 71.4 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

It seems that HERT openings are too balanced for Stockfish 16.1 (unless you don't mind lots of draws).

[Viz]

Water is wet.

[smatovic]

[...]
94% draws. Using Stockfish 11 against itself the draw rate lowers to 71%
[...]

Interesting, a graph of SF versions' draw rate over time would be fun.

--
Srdja

[MMarco]

[...]
94% draws. Using Stockfish 11 against itself the draw rate lowers to 71%
[...]

Interesting, a graph of SF versions' draw rate over time would be fun.

--
Srdja

Interestingly, old Stockfish 8 had a 77% draw rate, higher than Stockfish 11.

Code: Select all

Score of Stockfish_8 vs Stockfish_8_copy: 55 - 59 - 386 [0.496]
...      Stockfish_8 playing White: 39 - 22 - 189  [0.534] 250
...      Stockfish_8 playing Black: 16 - 37 - 197  [0.458] 250
...      White vs Black: 76 - 38 - 386  [0.538] 500
Elo difference: -2.8 +/- 14.5, LOS: 35.4 %, DrawRatio: 77.2 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

But with Stockfish 12, it increased to 87%.

Code: Select all

Score of Stockfish 12 vs Stockfish 12_copy: 42 - 25 - 433 [0.517]
...      Stockfish 12 playing White: 34 - 3 - 213  [0.562] 250
...      Stockfish 12 playing Black: 8 - 22 - 220  [0.472] 250
...      White vs Black: 56 - 11 - 433  [0.545] 500
Elo difference: 11.8 +/- 11.1, LOS: 98.1 %, DrawRatio: 86.6 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

Stockfish 14.1 is getting close to Stockfish 16.1 with 92%.

Code: Select all

Score of Stockfish 14.1 vs Stockfish 14.1_copy: 16 - 22 - 462 [0.494]
...      Stockfish 14.1 playing White: 9 - 4 - 237  [0.510] 250
...      Stockfish 14.1 playing Black: 7 - 18 - 225  [0.478] 250
...      White vs Black: 27 - 11 - 462  [0.516] 500
Elo difference: -4.2 +/- 8.4, LOS: 16.5 %, DrawRatio: 92.4 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished

[smatovic]

Thanks

--
Srdja

[Viz]

[...]
94% draws. Using Stockfish 11 against itself the draw rate lowers to 71%
[...]

Interesting, a graph of SF versions' draw rate over time would be fun.

--
Srdja

Interestingly, old Stockfish 8 had a 77% draw rate, higher than Stockfish 11.

Most likely an artifact of contempt which actually is enabled for both sides.
If you will set it to 0 by hand you should get higher drawrate for sf 11.

[MMarco]

[...]
94% draws. Using Stockfish 11 against itself the draw rate lowers to 71%
[...]

Interesting, a graph of SF versions' draw rate over time would be fun.

--
Srdja

Interestingly, old Stockfish 8 had a 77% draw rate, higher than Stockfish 11.

Most likely an artifact of contempt which actually is enabled for both sides.
If you will set it to 0 by hand you should get higher drawrate for sf 11.

Indeed. With contempt set to zero SF11 draw rate went from 71% to 80%. I also redid the test for SF12 with contempt set to zero.

Code: Select all

Score of Stockfish 11 vs Stockfish 11_copy: 52 - 48 - 400 [0.504]
...      Stockfish 11 playing White: 31 - 20 - 199  [0.522] 250
...      Stockfish 11 playing Black: 21 - 28 - 201  [0.486] 250
...      White vs Black: 59 - 41 - 400  [0.518] 500
Elo difference: 2.8 +/- 13.6, LOS: 65.5 %, DrawRatio: 80.0 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

Score of Stockfish 12 vs Stockfish 12_copy: 36 - 34 - 430 [0.502]
...      Stockfish 12 playing White: 28 - 7 - 215  [0.542] 250
...      Stockfish 12 playing Black: 8 - 27 - 215  [0.462] 250
...      White vs Black: 55 - 15 - 430  [0.540] 500
Elo difference: 1.4 +/- 11.4, LOS: 59.4 %, DrawRatio: 86.0 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
500 of 500 games finished.

Here are the corrected results:

SF8 --> 77.2%
SF11 --> 80.0%
SF12 --> 86.0%
SF14.1 -> 92.4%
SF16.1 -> 93.8%

[Viz]

I can't really recall if contempt did a lot for sf12, probably it did, but this results are kinda expected.
Contempt is also a huge factor of elo gains from version 9 to 11 where both versions - newer one and older one used contempt, so newer got double benefit from it.
So when you see "50 elo + gain" most likely with 0 contempt from both sides it will be 40 and with contempt only enabled for stronger version it will be 45, smth like this.
Since introduction of NNUE contempt more or less stopped working but 12 should still get benefit vs sf 11 because latter has contempt enabled by default (and contempt hurts you extremely hard vs a better positional player, and as you would guess even first nnue archs were completely superior to HCE, they managed to get like 50 elo while having a severe slowdown).
Since then it got removed so it never affected measured elo diffs between versions or against other engines.

[jefk]

there's also a 'perfect' book
https://sites.google.com/site/computers ... 2023-books

ofcourse chess is draw with perfect play.

Corroborating this with scientific reasoning (*) also is easy: if Black cannot be checkmated
in all situations, it must be (in a fundamental sense) a draw; (if you cannot force a mate for White
and the tree is widening, then you cannot force a mate for White later because the tree has
widened. In fact it's White who should avoid being checkmated:
1.g4 e5
2. f4 oops
2...Qh4 #

And for the math/comp guys : happy number crunching (with 'solving' chess)
-after 1.d4 there is Gruenfeld and it's a (theoretically) a draw (a certain Larry K knows this)
and after 1.Nf3! c5! 2.c4! you get symmetrical Engish and it's still fundamentally draw
(other options 1.h4?! and then winning in all situations, forget this balderdash)
ergo chess is 'weakly solved' (draw)

jef

(*) in such a way it's possible to 'prove' White cannot win, namely with a backward
induction reasoning method (**), but i'm not going to publish this here in detail.
For syzygt i suggest to doubt that the rabbit was faster than the

(**) any math kiddie as T.Tao or N. Elkies or so should be able to understand this;
at least Elkies (knowing about chess); and Tao later as well once i've published
it in more detail (end of this year, on researchgate or so)