Stockfish NNUE SV Tests

[mehmet123]

Raubfish vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 031021 x64 bmi2 : 2400 7 7 500 50.0 % 2400 94.8 %
2 Raubfisch X47d nn sl bmi2 : 2400 7 7 500 50.0 % 2400 94.8 %

Individual statistics:

1 Stockfish 031021 x64 bmi2 : 2400 500 (+ 13,=474,- 13), 50.0 %

Raubfisch X47d nn sl bmi2 : 500 (+ 13,=474,- 13), 50.0 %

2 Raubfisch X47d nn sl bmi2 : 2400 500 (+ 13,=474,- 13), 50.0 %

Stockfish 031021 x64 bmi2 : 500 (+ 13,=474,- 13), 50.0 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, 3 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/05aqjrzv ... 5.pgn/file

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 091021 x64 bmi2 : 2411 14 9 220 53.2 % 2389 93.6 %
2 Dragon 2.5.1 x64 avx2 HB1 : 2389 9 14 220 46.8 % 2411 93.6 %

Individual statistics:

1 Stockfish 091021 x64 bmi2 : 2411 220 (+ 14,=206,- 0), 53.2 %

Dragon 2.5.1 x64 avx2 HB1 : 220 (+ 14,=206,- 0), 53.2 %

2 Dragon 2.5.1 x64 avx2 HB1 : 2389 220 (+ 0,=206,- 14), 46.8 %

Stockfish 091021 x64 bmi2 : 220 (+ 0,=206,- 14), 46.8 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/wnuzhpgy ... 6.pgn/file

This is the best result for Dragon 2.5.1 against Stockfish Dev at this test conditions. In this test I changed some Dragon's engine parameters.

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 151021 x64 bmi2 : 2408 11 8 270 52.4 % 2392 94.4 %
2 Dragon 2.5.1 x64 avx2 HB2 : 2392 8 11 270 47.6 % 2408 94.4 %


Individual statistics:

1 Stockfish 151021 x64 bmi2 : 2408 270 (+ 14,=255,- 1), 52.4 %

Dragon 2.5.1 x64 avx2 HB2 : 270 (+ 14,=255,- 1), 52.4 %

2 Dragon 2.5.1 x64 avx2 HB2 : 2392 270 (+ 1,=255,- 14), 47.6 %

Stockfish 151021 x64 bmi2 : 270 (+ 1,=255,- 14), 47.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/zm3dbz2x ... 7.pgn/file


This is the new best result (-16 elo) for Dragon 2.5.1 against Stockfish Dev at this test conditions. I changed some Dragon settings again as in the previous test.

My old tests:
Elo difference between Stockfish 230921 and Dragon 2.5:
10 sec + 0.2 sec TC: + 86 elo
10 min + 0 sec TC: +26 elo

Elo difference between Stockfish 020521 and Dragon 2:
10 sec + 0.2 sec TC: + 118 elo
1 min+ 0.5 sec TC: +72 elo
5 min+ 0 sec TC:+56 elo


Dragon scales much better over time in matches up to at least 10 minutes of time than Stockfish. I'm guessing Dragon scales slightly better than Stockfish in matches longer than 10 minutes. It's no longer easy for Stockfish to beat Dragon in a minimum 100 game match (Tournament time control / 1 core).
If Dragon' s performance is at least as good as Stockfish on systems with high CPU numbers (16/32/64 ...cores), it won't be surprising if Dragon takes the title of the most powerful chess engine.

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 171021 x64 bmi2 : 2415 9 7 370 52.2 % 2392 95.1 %
2 Dragon 2.5.1 x64 avx2 HB3 : 2400 7 9 370 47.8 % 2408 95.1 %

Individual statistics:

1 Stockfish 171021 x64 bmi2 : 2415 370 (+ 17,=352,- 1), 52.2 %

Dragon 2.5.1 x64 avx2 HB3 : 370 (+ 17,=352,- 1), 52.2 %

2 Dragon 2.5.1 x64 avx2 HB3 : 2400 370 (+ 1,=352,- 17), 47.8 %

Stockfish 171021 x64 bmi2 : 370 (+ 1,=352,- 17), 47.8 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/u3cxf58o ... 8.pgn/file

This is the new best score (+15 elo) for Dragon against Stockfish for this test conditions. I changed some Dragon settings for this match again as in the previous tests.

[mehmet123]

For those who ask if these Dragon settings are special, it's just private for now. Because I haven't tried these settings in multiprocessor tests yet but I can send the HB3 settings privately if anyone wants to test them on multi-processor systems.
I doubt that these settings will give good results in rating lists. I saw this when I developed new settings for Lc0 chess engine targeting Stockfish last year. Some people claimed that Kayra settings that I developed for Lc0 are weaker than the standard settings. The reason for this was that Lc0 Kayra looked -5 elo weaker than the standard version in the FGRL rating list. In my tests Lc0 Kayra achieved 15-20 elo better results than the standard version against Stockfish. Looking at the performances against Stockfish in the FGRL list. It was revealed that Lc0 Kayra had 16-17 elo better results against the standard version like my tests.

Lc0 0.25.1 63305 - Stockfish NNUE SV20200725-1313 : 250 ( +3, =196, -51), % 40.4 :
Lc0 0.25.1 63305 - Stockfish 11 C0 : 250 ( +16, = 199, -35), %46.2

Lc0 0.25.1 63305 (Kayra4) - Stockfish NNUE SV20200725-1313 : 250 ( +7, = 199, -44), %42.6
Lc0 0.25.1 63305 (Kayra4) - Stockfish 11 C0 : 250 ( +28, =188, -34), %48.8
http://www.fastgm.de/h2h16-60.html#


It is possible to get good results for rating lists by playing Dragon with aggressive settings against weak engines but my purpose in these settings is to make adjustments where Dragon can get better results against Stockish.

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 181021 x64 bm2 : 2413 12 9 250 51.8 % 2400 93.2 %
2 Dragon 2.5.1 x64 avx2 HB5 : 2400 9 12 250 48.2 % 2413 93.2 %

Individual statistics:

1 Stockfish 181021 x64 bm2 : 2413 250 (+ 13,= 233,- 4), 51.8 %

Dragon 2.5.1 x64 avx2 HB5 : 250 (+ 4,= 233,- 13), 48.2 %

2 Dragon 2.5.1 x64 avx2 HB5 : 2400 250 (+ 4,= 233,- 13), 48.2 %

Stockfish 181021 x64 bm2 : 250 (+ 13,= 233,- 4), 51.8 %


Game Conditions: Fritz 17 Gui, 6 Cores (Core-i7 9750h), 5 min TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/qt4pqsht ... 9.pgn/file

This is the best performance of Dragon against Stockfish in my tests. I'm looking forward to the performance of Dragon's new net training against Lc0.

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 191021 x64 bmi2 : 2405 11 6 200 51.5 % 2395 97.0 %
2 Dragon 2.5.1 x64 avx2 HB4 : 2395 6 11 200 48.5 % 2405 97.0 %

Individual statistics:

1 Stockfish 191021 x64 bmi2 : 2405 200 (+ 6,=194,- 0), 51.5 %

Dragon 2.5.1 x64 avx2 HB4 : 200 (+ 6,=194,- 0), 51.5 %

2 Dragon 2.5.1 x64 avx2 HB4 : 2395 200 (+ 0,=194,- 6), 48.5 %

Stockfish 191021 x64 bmi2 : 200 (+ 0,=194,- 6), 48.5 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 min TC, Balsa 5 Moves Opening Book, 512 Mb Hash, Ponder Off
https://www.mediafire.com/file/4owgorsi ... 0.pgn/file

This is the new best score (-10 elo) for Dragon against Stockfish for this test conditions (min 200 games test).

[mehmet123]

Dragon 2.5.1 vs Stockfish Dev:

1: Stockfish 191021 x64 bmi2:2414 62.5 / 120 (+6 -1 =113)
2: Dragon 2.5.1 x64 avx2 HB4:2400 57.5 / 120 (+1 -6 =113)

Game Conditions: Fritz 17 Gui, 6 Cores (Core-i7 9750h), 5 min TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/3ryqnhrm ... 1.pgn/file

The performance of Dragon 2.5.1 x64 avx2 HB4 against Stockfish Dev is very close to Dragon 2.5.1 x64 avx2 HB5 at this test conditions.
-14 elo for Dragon 2.5.1 x64 avx2 HB4 against Stockfish Dev
-13 elo for Dragon 2.5.1 x64 avx2 HB5 against Stockfish Dev


HB4 Settings for Dragon:
https://www.mediafire.com/view/sb88yzie ... s.PNG/file

HB5 Settings for Dragon:
https://www.mediafire.com/view/nyl9bhb8 ... s.PNG/file

[mehmet123]

Koivisto 6.23 vs Stockfish 10:

Program Elo + - Games Score Av.Op. Draws

1 Koivisto 6.23 x64 avx2 : 2403 27 27 300 50.8 % 2397 53.0 %
2 Stockfish 10 x64 bmi2 : 2397 27 27 300 49.2 % 2403 53.0 %

Individual statistics:

1 Koivisto 6.23 x64 avx2 : 2403 300 (+ 73,=159,- 68), 50.8 %

Stockfish 10 x64 bmi2 : 300 (+ 73,=159,- 68), 50.8 %

2 Stockfish 10 x64 bmi2 : 2397 300 (+ 68,=159,- 73), 49.2 %

Koivisto 6.23 x64 avx2 : 300 (+ 68,=159,- 73), 49.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Koivisto 6.23 x64 avx2 ( "Ipman" compile)
https://www.mediafire.com/file/8bor1pes ... 2.pgn/file

Another great progress for Koivisto chess engine.


Previous Test:

Koivisto 6.16 vs Stockfish 10:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 10 x64 bmi2 : 2413 26 26 300 53.8 % 2387 55.7 %
2 Koivisto 6.16 x64 avx2 : 2387 26 26 300 46.2 % 2413 55.7 %

Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Koivisto 6.16 x64 avx2 ( "Ipman" compile)

[mehmet123]

Stockfish 14.1 vs Stockfish 14: (Fischer Random Chess)

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 14.1 x64 bmi2 : 2423 15 15 600 56.7 % 2377 69.0 %
2 Stockfish 14 x64 bmi2 : 2377 15 15 600 43.3 % 2423 69.0 %

Individual statistics:

1 Stockfish 14.1 x64 bmi2 : 2423 600 (+133,=414,- 53), 56.7 %

Stockfish 14 x64 bmi2 : 600 (+133,=414,- 53), 56.7 %

2 Stockfish 14 x64 bmi2 : 2377 600 (+ 53,=414,-133), 43.3 %

Stockfish 14.1 x64 bmi2 : 600 (+ 53,=414,-133), 43.3 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Chess960 3 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/dm7pl1e3 ... 3.pgn/file