Stockfish NNUE SV Tests

[mehmet123]

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 : 2466 13 12 1300 68.2 % 2334 53.8 %
2 Stockfish 11 x64 bmi2 : 2334 12 13 1300 31.8 % 2466 53.8 %

Individual statistics:

1 Cfish NNUE SV 1035 : 2466 1300 (+536,=700,- 64), 68.2 %

Stockfish 11 x64 bmi2 : 1300 (+536,=700,- 64), 68.2 %

2 Stockfish 11 x64 bmi2 : 2334 1300 (+ 64,=700,-536), 31.8 %

Cfish NNUE SV 1035 : 1300 (+ 64,=700,-536), 31.8 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation:Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)
http://www.mediafire.com/file/okvac9on2 ... 3.pgn/file

Frankly ı expected a much better performance after its performance against Stockfish 140820 (+130 elo) at this time control (1 min + 0.5 sec)

[mehmet123]

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 : 2466 13 12 1200 68.1 % 2334 56.5 %
2 Stockfish 11 x64 bmi2 : 2334 12 13 1200 31.9 % 2466 56.5 %

Individual statistics:

1 Cfish NNUE SV 1035 : 2466 1200 (+478,=678,- 44), 68.1 %

Stockfish 11 x64 bmi2 : 1200 (+478,=678,- 44), 68.1 %

2 Stockfish 11 x64 bmi2 : 2334 1200 (+ 44,=678,-478), 31.9 %

Cfish NNUE SV 1035 : 1200 (+ 44,=678,-478), 31.9 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, SuperGM 4 moves 500 Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)
http://www.mediafire.com/file/owq8ng3pv ... 4.pgn/file

[mehmet123]

Cfish NNUE SV 1035 vs Stockfish 11:
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)

+132 elo against Stockfish 11 with Balsa 5 Move Opening Book at 1 min + 0.5 sec TC (1300 games)
+132 elo against Stockfish 11 with SuperGM 4 moves 500 Opening Book at 1 min + 0.5 sec TC (1200 games)

The performance of Stockfish Dev NNUE against Stockfish 11 is +123 elo for now.
https://tests.stockfishchess.org/tests

[mehmet123]

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 (Hybrid) : 2471 12 11 1421 69.2 % 2329 55.7 %
2 Stockfish 11 x64 bmi2 : 2329 11 12 1421 30.8 % 2471 55.7 %

Individual statistics:

1 Cfish NNUE SV 1035 (Hybrid): 2471 1421 (+588,=792,- 41), 69.2 %

Stockfish 11 x64 bmi2 : 1421 (+588,=792,- 41), 69.2 %

2 Stockfish 11 x64 bmi2 : 2329 1421 (+ 41,=792,-588), 30.8 %

Cfish NNUE SV 1035 (Hybrid) : 1421 (+ 41,=792,-588), 30.8 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile /Member of The Outskirt Chess Forum )
http://www.mediafire.com/file/aif2qrkoj ... 6.pgn/file

Great performance (+142 elo) against Stockfish 11 x64 bmi2.
The performance of Cfish NNUE SV 1035 (Hybrid) is +10 elo better than Cfish NNUE SV 1035 (Pure).

[mehmet123]

10 years Of Chess Engines Test:

For now Cfish is the strongest chess engine. Ivanhoe was the one of the strongest chess engine of 2010

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2576 24 95 200 96.5 % 2000 6.0 %
2 IvanHoe T0.5.4.1 x64 : 2000 95 24 200 3.5 % 2576 6.0 %

Individual statistics:

1 Cfish 190820 x64 bmi2 : 2576 200 (+187,= 12,- 1), 96.5 %

IvanHoe T0.5.4.1 x64 : 200 (+187,= 12,- 1), 96.5 %

2 IvanHoe T0.5.4.1 x64 : 2000 200 (+ 1,= 12,-187), 3.5 %

Cfish 190820 x64 bmi2 : 200 (+ 1,= 12,-187), 3.5 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile/Member of The Outskirt Chess Forum) //SV 20200814-1035 Net
http://www.mediafire.com/file/1irue4f5z ... 7.pgn/file

Elo difference is great ( +576 elo)
The power change of chess programs over the last 10 years is incredible. SV Nets have an important role in the formation of this difference.

[mehmet123]

10 years Of Chess Engines Test:

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2576 24 95 200 96.5 % 2000 6.0 %
2 IvanHoe T0.5.4.1 x64 : 2000 95 24 200 3.5 % 2576 6.0 %

Individual statistics:

1 Cfish 190820 x64 bmi2 : 2576 200 (+187,= 12,- 1), 96.5 %

IvanHoe T0.5.4.1 x64 : 200 (+187,= 12,- 1), 96.5 %

2 IvanHoe T0.5.4.1 x64 : 2000 200 (+ 1,= 12,-187), 3.5 %

Cfish 190820 x64 bmi2 : 200 (+ 1,= 12,-187), 3.5 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile) //SV 20200814-1035 Net
http://www.mediafire.com/file/6l7k185y9 ... 5.pgn/file

I made another test with 1 min + 0.5 sec TC. The result is same ( +576 elo)

[mehmet123]

Cfish lost 2 out of 400 games against Ivanhoe. 2 lost games are the same Sicilian Variant.

[pgn]1.e4 c5 2. Nc3 Nc6 3. Nf3 e5 4. Bc4 Be7 5. d3 d6 [/pgn]
Ivanhoe performed very well in this variant beyond its strength although it was too weak for Cfish.
I made a new match with all games starting with this variant.

1. e4 c5 2. Nc3 Nc6 3. Nf3 e5 4. Bc4 Be7 5. d3 d6 Opening Match

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2254 56 54 200 81.2 % 2000 12.5 %
2 IvanHoe T0.5.4.1 x64 : 2000 54 56 200 18.8 % 2254 12.5 %

Score of Cfish 190820 x64 bmi2 vs IvanHoe T0.5.4.1 x64 Game_Mode: 150 - 25 - 25 [0.813]
... Cfish 190820 x64 bmi2 playing White: 57 - 25 - 18 [0.660] 100
... Cfish 190820 x64 bmi2 playing Black: 93 - 0 - 7 [0.965] 100
Elo difference: 254.7 +/- 55.9, LOS: 100.0 %, DrawRatio: 12.5 %

Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile) //SV 20200814-1035 Net
http://www.mediafire.com/file/flen98ckm ... 6.pgn/file

Ivanhoe won 25 games with black at this variant. But Ivanhoe never managed to win games with white at this variant.
The performance of Cfish with whites in this variant is well below expectations.(% 66)
But it's performance with black in this variant is very well (% 96.5)

[mehmet123]

Cfish vs Stockfish Match:

Program Elo + - Games Score Av.Op. Draws

1 Cfish 230820 x64 bmi2 : 2405 9 8 1000 51.5 % 2395 84.3 %
2 Stockfish 240820 x64 bmi2 : 2395 8 9 1000 48.4 % 2405 84.3 %

Individual statistics:

1 Cfish 230820 x64 bmi2: 2405 1000 (+ 94,=843,- 63), 51.5 %

Stockfish 240820 x64 bmi2 : 1000 (+ 94,=843,- 63), 51.5 %

2 Stockfish 240820 x64 bmi2 : 2395 1000 (+ 63,=843,- 94), 48.5 %

Cfish 230820 x64 bmi2 : 1000 (+ 63,=843,- 94), 48.4 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation1: Cfish 230820 x64 BMI2 mingw 10.2 (ChessMan compile) // Default Net
Compilation2: Stockfish 240820 x64 bmi2 (Official compile)// Default Net
http://www.mediafire.com/file/gjjn3dfeq ... 0.pgn/file

Cfish is a bit behind in the code update but managed to beat Stockfish.
+10 elo difference over Stockfish is great success.

[mehmet123]

Test of SV 20200824-1705

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1035 : 2423 12 12 3000 56.5 % 2377 3.0 %
2 Stockfish NNUE SV 1705 : 2377 12 12 3000 43.5 % 2423 3.0 %

Individual statistics:

1 Stockfish NNUE SV 1035 : 2423 3000 (+1650,= 90,-1260), 56.5 %

Stockfish NNUE SV 1705 : 3000 (+1650,= 90,-1260), 56.5 %

2 Stockfish NNUE SV 1705 : 2377 3000 (+1260,= 90,-1650), 43.5 %

Stockfish NNUE SV 1035 : 3000 (+1260,= 90,-1650), 43.5 %

Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 10 kn/move, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 240820 BMI2 mingw 10.2(ChessMan compile)
http://www.mediafire.com/file/4ci326l9j ... 1.pgn/file

At 10 kn/move match SV 1705 net is very weaker ( - 46 elo) than SV 1035 net.

[mehmet123]

Test of SV 20200824-1705

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1035 : 2407 10 10 3000 52.1 % 2393 35.8 %
2 Stockfish NNUE SV 1705 : 2393 10 10 3000 47.9 % 2407 35.8 %

Individual statistics:

1 Stockfish NNUE SV 1035 : 2407 3000 (+1026,=1074,-900), 52.1 %

Stockfish NNUE SV 1705 : 3000 (+1026,=1074,-900), 52.1 %

2 Stockfish NNUE SV 1705 : 2393 3000 (+900,=1074,-1026), 47.9 %

Stockfish NNUE SV 1035 : 3000 (+900,=1074,-1026), 47.9 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 20 kn/move, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 240820 BMI2 mingw 10.2(ChessMan compile)
http://www.mediafire.com/file/jvca3w830 ... 2.pgn/file

At 20 kn/move match the performance of SV 1705 net is much better ( -14 elo) according to its 10 kn/ move match performance.