Stockfish 1 - Stockfish 7 in Arasan testsuite

[Jouni]

As we all know testsuites are almost useless to detect any engine progress. Are they? Arasan suite v19 gives surprisingly steady
increase even with SMP randomness. Solved positions with 30 s and 4 cores:

SF 1 101
SF 2 103
SF 3 119
SF 4 124
SF 5 138
SF 6 145
SF 7 156

And SF 24.9. - surprise, surprise 159!

[Sean Evans]

Not difficult when the testsuites are embedded in the engines database.

[Vinvin]

Not difficult when the testsuites are embedded in the engines database.

Can you point where this database would be here : https://github.com/official-stockfish/S ... master/src ?

[Jouni]

Actually if I remember correctly there was a commercial engine with embedded/hardcoded testpositions long time ago. Does anybody remember that case? Fritz? Nimzo?

[tpoppins]

The only one I know of that did that is Strelka 5.5 (2012).

[Sean Evans]

Actually if I remember correctly there was a commercial engine with embedded/hardcoded testpositions long time ago. Does anybody remember that case? Fritz? Nimzo?

MChess Pro had an embedded hidden opening book prepared specifically for Chess Genius in this game:

[pgn]
[Event "WMCCC 1995"]
[Site "Paderborn, Germany"]
[Date "1995.10.15"]
[Round "12 (playoff)"]
[White "MChess Pro"]
[Black "Chess Genius"]
[Result "1-0"]

1.e4 d5 2.exd5 Qxd5 3.Nc3 Qa5 4.d4 Nf6 5.Nf3 Bf5 6.Bc4 e6 7.O-O c6 8.Re1 Nbd7
9.h3 Bd6 10.Bd2 Qc7 11.Nh4 Bg6 12.Nxg6 hxg6 13.Qf3 O-O-O 14.a3 Nb6 15.Bf1 Rh4
16.g3 Rh5 17.Ne4 Nxe4 18.Qxe4 Rdh8 19.c4 Bxg3 20.fxg3 Qxg3+ 21.Qg2 Rxh3 22.Qxg3
Rxg3+ 23.Kf2 Rb3 24.Bc1 Rh4 25.Re3 Rxe3 26.Kxe3 c5 27.dxc5 Nxc4+ 28.Kf2 f6 29.b4
g5 30.Bg2 Ne5 31.Bd2 Rc4 32.Ra2 Kc7 33.Bh3 Kd7 34.Bf1 Rh4 35.Bg2 Kc7 36.Ra1 Rc4
37.Ke2 Rc2 38.Be4 Rb2 39.Kd1 Rb3 40.Ke2 Rh3 41.Rh1 Rxh1 42.Bxh1 Nc4 43.Bc1 g4
44.Bg2 b6 45.Kd3 Ne5+ 46.Ke4 Ng6 47.Bf4+ e5 48.Bg3 bxc5 49.bxc5 Ne7 50.Bf1 Kc6
51.Bf2 g3 52.Bg1 g6 53.a4 a5 54.Bb5+ Kc7 55.Kf3 Nf5 56.Bd3 Ne7 57.Kxg3 Kc6
58.Be4+ Kd7 59.Ba8 Ke6 60.Kg4 Nd5 61.Bb7 Nb4 62.Be3 Kd7 63.Bd2 Nc6 64.Ba6 Nb4
65.Bb5+ Ke7 66.Bxb4 axb4 67.a5 Kd8 68.a6 Kc7 69.Bc4 f5+ 70.Kg5 f4 71.Bd5 b3
72.Bxb3 f3 73.Bc4 f2 74.Bb5 e4 75.Kf4 g5+ 76.Kxe4 Kb8 77.c6 Kc7 1-0[/pgn]

[bob]

Actually if I remember correctly there was a commercial engine with embedded/hardcoded testpositions long time ago. Does anybody remember that case? Fritz? Nimzo?

Rebel...

[Rebel]

Actually if I remember correctly there was a commercial engine with embedded/hardcoded testpositions long time ago. Does anybody remember that case? Fritz? Nimzo?

Not Fritz, nor Nimzo but this ugly guy.

[Adam Hair]

[voyagerOne]

As we all know testsuites are almost useless to detect any engine progress. Are they? Arasan suite v19 gives surprisingly steady
increase even with SMP randomness. Solved positions with 30 s and 4 cores:

SF 1 101
SF 2 103
SF 3 119
SF 4 124
SF 5 138
SF 6 145
SF 7 156

And SF 24.9. - surprise, surprise 159!

Hey JU-
Do you have results for single core?