Stockfish NNUE SV Tests

[mehmet123]

Dragon by Komodo vs. Stockfish 12 NNUE:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 12 x64 bmi2 : 2409 13 12 300 52.7 % 2391 90.0 %
2 Dragon x64 avx2 : 2391 12 13 300 47.3 % 2409 90.0 %

Individual statistics:

1 Stockfish 12 x64 bmi2 : 2409 300 (+ 23,=270,- 7), 52.7 %

Dragon x64 avx2 : 300 (+ 23,=270,- 7), 52.7 %

2 Dragon x64 avx2 : 2391 300 (+ 7,=270,- 23), 47.3 %

Stockfish 12 x64 bmi2 : 300 (+ 7,=270,- 23), 47.3 %


Game Conditions: Cutechess Gui, 6 Cores (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book,1024 Mb Hash, Ponder Off
Contempt :Dragon by Komodo:0, Stockfish 12: Default (24)
http://www.mediafire.com/file/iy0yu22v9 ... 0.pgn/file

The performance of Dragon is impressive at 6 cores, 1 min + 0.5 TC match. Dragon is only -18 elo ahead of Stockfish 12. At 1 core , 10 sec+ 0.5 sec TC test Dragon is -46 elo ahead of Stockfish 12.
In the match where engines calculated 25 times more moves, Dragon reduced the elo difference with Stockfish 12 by 28 elo.
Dragon chess engine looks like it will be a serious rival to Stockfish Dev. in the TCEC competition.

[mehmet123]

Dragon by Komodo vs. Stockfish 12 NNUE (Fischer Random Chess):

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 12 x64 bmi2 : 2416 16 16 1000 54.7 % 2384 46.6 %
2 Dragon x64 avx2 : 2384 16 16 1000 45.3 % 2416 46.6 %

Individual statistics:

1 Stockfish 12 x64 bmi2 : 2416 1000 (+314,=466,-220), 54.7 %

Dragon x64 avx2 : 1000 (+314,=466,-220), 54.7 %

2 Dragon x64 avx2 : 2384 1000 (+220,=466,-314), 45.3 %

Stockfish 12 x64 bmi2 : 1000 (+220,=466,-314), 45.3 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Chess960 3 Moves Opening Book, 64 Mb Hash, Ponder Off
Contempt :Dragon by Komodo:0, Stockfish 12: Default (24)
http://www.mediafire.com/file/6j9n9jw2j ... 3.pgn/file

At FRC match the performance of Dragon is better than Standart Chess at this conditions (1 core, 10 sec + 0.2 sec TC). At standart chess match the performance of Dragon is -46 elo, at FRC match the performance of Dragon is -32 elo.

[Vinvin]

Any hope you test Honey-12-R1 and Bluefish-12-R1 ?

2 other very strong SF evolutions to test : Honey-12-R1 and Bluefish-12-R1
At the bottom of the page : https://github.com/MichaelB7/Stockfish/ ... /tag/v12r1
May be you have to add the latest nn-file from here : https://tests.stockfishchess.org/nns

[mehmet123]

Dragon by Komodo vs. Stockfish 12 NNUE:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 12 x64 bmi2 : 2407 16 13 160 51.9 % 2393 92.5 %
2 Dragon x64 avx2 : 2393 13 16 160 48.1 % 2407 92.5 %

Individual statistics:

1 Stockfish 12 x64 bmi2 : 2407 160 (+ 9,=148,- 3), 51.9 %

Dragon x64 avx2 : 160 (+ 9,=148,- 3), 51.9 %

2 Dragon x64 avx2 : 2393 160 (+ 3,=148,- 9), 48.1 %

Stockfish 12 x64 bmi2 : 160 (+ 3,=148,- 9), 48.1 %


Game Conditions: Cutechess Gui, 6 Cores (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book,1024 Mb Hash, Ponder Off
Contempt :Dragon by Komodo:0, Stockfish 12: Default (24)
http://www.mediafire.com/file/u6w95856o ... 1.pgn/file

The performance of Dragon is -14 elo according to Stockfish 12 at 6 cores, 2 min + 0.5 sec TC
The performance of Dragon is -18 elo according to Stockfish 12 at 6 cores, 1 min + 0.5 sec TC
The performance of Dragon is -46 elo according to Stockfish 12 at 1 core, 10 sec + 0.2 sec TC

[mehmet123]

Honey vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 101120 x64 bmi2 : 2405 12 12 600 51.5 % 2395 81.7 %
2 Honey 12- R1 x64 bmi2 : 2395 12 12 600 48.5 % 2405 81.7 %

Individual statistics:

1 Stockfish 101120 x64 bmi2 : 2405 600 (+ 64,=490,- 46), 51.5 %

Honey 12- R1 x64 bmi2 : 600 (+ 64,=490,- 46), 51.5 %

2 Honey 12- R1 x64 bmi2 : 2395 600 (+ 46,=490,- 64), 48.5 %

Stockfish 101120 x64 bmi2 : 600 (+ 46,=490,- 64), 48.5 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Only Stockfish plays default net.
http://www.mediafire.com/file/vi6lw6doe ... 4.pgn/file

The performance of Honey ( -10 elo) is very good because the last version of Honey was published before 1.5 months than Stockfish Dev.

[mehmet123]

Honey vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 101120 x64 bmi2 : 2404 10 10 600 51.2 % 2396 86.8 %
2 Honey 12-R1 : 2396 10 10 600 48.8 % 2404 86.8 %

Individual statistics:

1 Stockfish 101120 x64 bmi2 : 2404 600 (+ 47,=521,- 32), 51.2 %

Honey 12-R1 : 600 (+ 47,=521,- 32), 51.2 %

2 Honey 12-R1 : 2396 600 (+ 32,=521,- 47), 48.8 %

Stockfish 101120 x64 bmi2 : 600 (+ 32,=521,- 47), 48.8 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Only Stockfish plays default net.
http://www.mediafire.com/file/z0hewkgcq ... 0.pgn/file

The performance of Honey is a little better (-8 elo) at 30 sec + 0.5 sec TC.

[mehmet123]

NNUE vs Opening Book:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish Polyglot 031120 NNUE : 2415 15 15 411 54.4 % 2385 79.1 %
2 Stockfish Polyglot 031120 : 2385 15 15 411 45.6 % 2415 79.1 %

Individual statistics:

1 Stockfish Polyglot 031120 NNUE: 2415 411 (+ 61,=325,- 25), 54.4 %

Stockfish Polyglot 031120 : 411 (+ 61,=325,- 25), 54.4 %

2 Stockfish Polyglot 031120 : 2385 411 (+ 25,=325,- 61), 45.6 %

Stockfish Polyglot 031120 NNUE: 411 (+ 25,=325,- 61), 45.6 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min TC, 64 Mb Hash, Ponder Off
Stockfish Polyglot 031120 NNUE played with NNUE (nn-cb26f10b1fd9)
Stockfish Polyglot 031120 played with Cevdet-X Opening Book, without NNUE
http://www.mediafire.com/file/ri62zr8pp ... 0.pgn/file

[mehmet123]

NNUE vs Opening Book:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish Polyglot 031120 NNUE : 2420 13 13 480 55.6 % 2380 81.7 %
2 Stockfish Polyglot 031120 : 2380 13 13 480 44.4 % 2420 81.7 %

Individual statistics:

1 Stockfish Polyglot 031120 NNUE: 2420 480 (+ 71,=392,- 17), 55.6 %

Stockfish Polyglot 031120 : 480 (+ 71,=392,- 17), 55.6 %

2 Stockfish Polyglot 031120 : 2380 480 (+ 17,=392,- 71), 44.4 %

Stockfish Polyglot 031120 NNUE: 480 (+ 17,=392,- 71), 44.4 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 3 min TC, 128 Mb Hash, Ponder Off
Stockfish Polyglot 031120 NNUE played with NNUE (nn-cb26f10b1fd9)
Stockfish Polyglot 031120 played with Cevdet-X Opening Book(by Cevdet SARI), without NNUE
http://www.mediafire.com/file/loef8cyjj ... 1.pgn/file

[mehmet123]

Opening Book vs. Without Opening Book:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish Polyglot 031120 (Cevdet-X) : 2417 11 10 500 54.8 % 2383 88.0 %
2 Stockfish Polyglot 031120 : 2383 10 11 500 45.2 % 2417 88.0 %

Individual statistics:

1 Stockfish Polyglot 031120 (Cevdet-X): 2417 500 (+ 54,=440,- 6), 54.8 %

Stockfish Polyglot 031120 : 500 (+ 54,=440,- 6), 54.8 %

2 Stockfish Polyglot 031120 : 2383 500 (+ 6,=440,- 54), 45.2 %

Stockfish Polyglot 031120 (Cevdet-X): 500 (+ 6,=440,- 54), 45.2 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min TC, 128 Mb Hash, Ponder Off
Stockfish Polyglot 031120 Cevdet- X played with Cevdet-X Opening Book(by Cevdet SARI) and with NNUE (nn-cb26f10b1fd9)
Stockfish Polyglot 031120 played with with NNUE (nn-cb26f10b1fd9) ,without Opening Book
http://www.mediafire.com/file/wz21xnrm5 ... 2.pgn/file

[mehmet123]

Cfish vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Cfish 301120 x64 bmi2 : 2404 8 8 600 51.1 % 2396 91.5 %
2 Stockfish 291120 x64 bmi2 : 2396 8 8 600 48.9 % 2404 91.5 %

Individual statistics:

1 Cfish 301120 x64 bmi2 : 2404 600 (+ 32,=549,- 19), 51.1 %

Stockfish 291120 x64 bmi2 : 600 (+ 32,=549,- 19), 51.1 %

2 Stockfish 291120 x64 bmi2 : 2396 600 (+ 19,=549,- 32), 48.9 %

Cfish 301120 x64 bmi2 : 600 (+ 19,=549,- 32), 48.9 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
Cfish compile:Cfish 301120 x64 E BMI2 mingw10 (ChessMan compile)//Default net
http://www.mediafire.com/file/wga1aunu5 ... 1.pgn/file