Stockfish NNUE SV Tests

[mehmet123]

Stockfish 14 vs Stockfisk 13:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 14 x64 bmi2 : 2428 11 11 1000 58.1 % 2372 72.7 %
2 Stockfish 13 x64 bmi2 : 2372 11 11 1000 41.9 % 2428 72.7 %

Individual statistics:

1 Stockfish 14 x64 bmi2 : 2428 1000 (+217,=727,- 56), 58.0 %

Stockfish 13 x64 bmi2 : 1000 (+217,=727,- 56), 58.1 %

2 Stockfish 13 x64 bmi2 : 2372 1000 (+ 56,=727,-217), 42.0 %

Stockfish 14 x64 bmi2 : 1000 (+ 56,=727,-217), 41.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/826pqr1y ... 7.pgn/file

Stockfish 14 is very strong at ultra fast time (10 sec + 0.2 sec) control matches.

[mehmet123]

Stockfish 14 vs Stockfisk 13:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 14 x64 bmi2 : 2417 8 7 1000 54.8 % 2383 87.0 %
2 Stockfish 13 x64 bmi2 : 2383 7 8 1000 45.2 % 2417 87.0 %

Individual statistics:

1 Stockfish 14 x64 bmi2 : 2417 1000 (+113,=870,- 17), 54.8 %

Stockfish 13 x64 bmi2 : 1000 (+113,=870,- 17), 54.8 %

2 Stockfish 13 x64 bmi2 : 2383 1000 (+ 17,=870,-113), 45.2 %

Stockfish 14 x64 bmi2 : 1000 (+ 17,=870,-113), 45.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/1m24lq7a ... 8.pgn/file

[mehmet123]

ChessMan compile vs Official compile:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 14 x64 bmi2 C : 2401 7 7 1200 50.3 % 2399 85.9 %
2 Stockfish 14 x64 bmi2 O : 2399 7 7 1200 49.7 % 2401 85.9 %

Individual statistics:

1 Stockfish 14 x64 bmi2 C : 2401 1200 (+ 88,=1031,- 81), 50.3 %

Stockfish 14 x64 bmi2 O : 1200 (+ 88,=1031,- 81), 50.3 %

2 Stockfish 14 x64 bmi2 O : 2399 1200 (+ 81,=1031,- 88), 49.7 %

Stockfish 14 x64 bmi2 C : 1200 (+ 81,=1031,- 88), 49.7 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
Stockfish 14 x64 bmi2 C:ChessMan compile (https://pixeldrain.com/u/cAfodUFh)
Stockfish 14 x64 bmi2 O:Official compile (https://stockfishchess.org/download/windows/)
https://www.mediafire.com/file/ateyd24j ... 9.pgn/file

It would make more sense to do a speed test for Stockfish versions as different versions may be faster on different systems.

[mehmet123]

Koivisto 6.0 vs Stockfish 8:

Program Elo + - Games Score Av.Op. Draws

1 Koivisto 6.0 x64 avx2 : 2411 20 20 500 53.1 % 2389 57.0 %
2 Stockfish 8 x64 bmi2 : 2389 20 20 500 46.9 % 2411 57.0 %

Individual statistics:

1 Koivisto 6.0 x64 avx2 : 2411 500 (+123,=285,- 92), 53.1 %

Stockfish 8 x64 bmi2 : 500 (+123,=285,- 92), 53.1 %

2 Stockfish 8 x64 bmi2 : 2389 500 (+ 92,=285,-123), 46.9 %

Koivisto 6.0 x64 avx2 : 500 (+ 92,=285,-123), 46.9 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/ddvy797i ... 4.pgn/file

Koivisto chess engine is one of the few chess engines to dominate Stockfish 8.

[mehmet123]

Koivisto 6 vs Stockfish 8:

Program Elo + - Games Score Av.Op. Draws

1 Koivisto 6.0 x64 avx2 : 2415 21 21 600 54.4 % 2385 45.2 %
2 Stockfish 8 x64 bmi2 : 2385 21 21 600 45.6 % 2415 45.2 %

Individual statistics:

1 Koivisto 6.0 x64 avx2 : 2415 600 (+191,=271,-138), 54.4 %

Stockfish 8 x64 bmi2 : 600 (+191,=271,-138), 54.4 %

2 Stockfish 8 x64 bmi2 : 2385 600 (+138,=271,-191), 45.6 %

Koivisto 6.0 x64 avx2 : 600 (+138,=271,-191), 45.6 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/xoosj8fy ... 5.pgn/file

Koivisto 6 was able to dominate not only Stockfish 8 but also Houdini 6.


Koivisto vs Houdini:

Program Elo + - Games Score Av.Op. Draws

1 Koivisto 6.0 x64 avx2 : 2402 19 19 500 50.6 % 2398 59.2 %
2 Houdini 6 Pro x64 pext : 2398 19 19 500 49.4 % 2402 59.2 %

Individual statistics:

1 Koivisto 6.0 x64 avx2 : 2402 500 (+105,=296,- 99), 50.6 %

Houdini 6 Pro x64 pext : 500 (+105,=296,- 99), 50.6 %

2 Houdini 6 Pro x64 pext : 2398 500 (+ 99,=296,-105), 49.4 %

Koivisto 6.0 x64 avx2 : 500 (+ 99,=296,-105), 49.4 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/r11q49or ... 7.pgn/file

[mehmet123]

Stockfish miniNNUE vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 130721 x64 bmi2 : 2401 7 7 700 50.2 % 2399 93.0 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 7 7 700 49.8 % 2401 93.0 %

Individual statistics:

1 Stockfish 130721 x64 bmi2 : 2401 700 (+ 26,=651,- 23), 50.2 %

Stockfish miniNNUE 170721 bmi2 : 700 (+ 26,=651,- 23), 50.2 %

2 Stockfish miniNNUE 170721 bmi2 : 2400 700 (+ 23,=651,- 26), 49.8 %

Stockfish 130721 x64 bmi2 : 700 (+ 23,=651,- 26), 49.8 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/4zgtt4t2 ... 8.pgn/file

The performance of Stockfish miniNNUE against Stockfish Dev is very good considering it plays with a Net that is about 1/4 the size of Default Net.

[mehmet123]

Stockfish miniNNUE vs Stockfish Dev (Fischer Random Chess):

Program Elo + - Games Score Av.Op. Draws

1 Stockfish miniNNUE 170721 bmi2 : 2409 12 12 700 52.6 % 2391 79.3 %
2 Stockfish 230721 x64 bmi2 : 2391 12 12 700 47.4 % 2409 79.3 %

Individual statistics:

1 Stockfish miniNNUE 170721 bmi2 : 2409 700 (+ 91,=555,- 54), 52.6 %

Stockfish 230721 x64 bmi2 : 700 (+ 91,=555,- 54), 52.6 %

2 Stockfish 230721 x64 bmi2 : 2391 700 (+ 54,=555,- 91), 47.4 %

Stockfish miniNNUE 170721 bmi2 : 700 (+ 54,=555,- 91), 47.4 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 2 min + 0.5 sec TC, Chess960 3 Moves Opening Book, 128 Mb Hash, Ponder Off
https://www.mediafire.com/file/v0lzyjtw ... 9.pgn/file

The performance of Stockfish miniNNUE against Stockfish Dev is great (+18 elo) at FRC match.

[Modern Times]

The performance of Stockfish miniNNUE against Stockfish Dev is great (+18 elo) at FRC match.

Interesting result !

[mehmet123]

Stockfish miniNNUE vs Stockfish Dev:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish 240721 x64 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %
2 Stockfish miniNNUE 170721 bmi2 : 2400 5 5 800 50.0 % 2400 96.5 %

Individual statistics:

1 Stockfish 240721 x64 bmi2 : 2400 800 (+ 14,=772,- 14), 50.0 %

Stockfish miniNNUE 170721 bmi2: 800 (+ 14,=772,- 14), 50.0 %

2 Stockfish miniNNUE 170721 bmi2: 2400 800 (+ 14,=772,- 14), 50.0 %

Stockfish 240721 x64 bmi2 : 800 (+ 14,=772,- 14), 50.0 %


Game Conditions: Arena Gui, 6 Cores (Core-i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Moves Opening Book, 1024 Mb Hash, Ponder Off
https://www.mediafire.com/file/r8w74ybt ... 1.pgn/file


After 800 games 1 min+ 0.5 sec time control match at 6 cores Stockish Dev can't beat Stockfish miniNNUE 170721. After 770 games Stockfish miniNNUE 170721 was ahead of with a score of 386-384 (+ 14,=744,- 12)

We only know which NNUE nets are best for 1 min + 0.6 sec (1 core) and faster time control matches. We don't have enough information about which NNUE nets are strong in games longer than 1 min + 0.6 sec time control matches or in matches where Stockfish chess engines are played with multiprocessors

[mehmet123]

Fire vs Stockfish:

Program Elo + - Games Score Av.Op. Draws

1 Fire 8.NN MC x64 bmi2 : 2431 15 15 1100 58.8 % 2369 48.9 %
2 Stockfish 310720 x64 bmi2 : 2369 15 15 1100 41.2 % 2431 48.9 %

Individual statistics:

1 Fire 8.NN MC x64 bmi2 : 2431 1100 (+378,=538,-184), 58.8 %

Stockfish 310720 x64 bmi2 : 1100 (+378,=538,-184), 58.8 %

2 Stockfish 310720 x64 bmi2 : 2369 1100 (+184,=538,-378), 41.2 %

Fire 8.NN MC x64 bmi2 : 1100 (+184,=538,-378), 41.2 %


Game Conditions: Cutechess Gui, 1 Core (Core-i7 9750h), 10 sec + 0.2 sec TC, Balsa 5 Moves Opening Book, 64 Mb Hash, Ponder Off
https://www.mediafire.com/file/c2uj4isv ... 4.pgn/file

Stockfish 310720 is the last Stockfish Dev version without NNUE evaluation.