Stockfish NNUE SV Tests

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Aug 19, 2020 4:48 pm

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 : 2466 13 12 1300 68.2 % 2334 53.8 %
2 Stockfish 11 x64 bmi2 : 2334 12 13 1300 31.8 % 2466 53.8 %

Individual statistics:

1 Cfish NNUE SV 1035 : 2466 1300 (+536,=700,- 64), 68.2 %

Stockfish 11 x64 bmi2 : 1300 (+536,=700,- 64), 68.2 %

2 Stockfish 11 x64 bmi2 : 2334 1300 (+ 64,=700,-536), 31.8 %

Cfish NNUE SV 1035 : 1300 (+ 64,=700,-536), 31.8 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation:Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)
http://www.mediafire.com/file/okvac9on2 ... 3.pgn/file

Frankly ı expected a much better performance after its performance against Stockfish 140820 (+130 elo) at this time control (1 min + 0.5 sec)

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Aug 19, 2020 5:05 pm

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 : 2466 13 12 1200 68.1 % 2334 56.5 %
2 Stockfish 11 x64 bmi2 : 2334 12 13 1200 31.9 % 2466 56.5 %

Individual statistics:

1 Cfish NNUE SV 1035 : 2466 1200 (+478,=678,- 44), 68.1 %

Stockfish 11 x64 bmi2 : 1200 (+478,=678,- 44), 68.1 %

2 Stockfish 11 x64 bmi2 : 2334 1200 (+ 44,=678,-478), 31.9 %

Cfish NNUE SV 1035 : 1200 (+ 44,=678,-478), 31.9 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, SuperGM 4 moves 500 Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)
http://www.mediafire.com/file/owq8ng3pv ... 4.pgn/file

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Aug 19, 2020 5:22 pm

Cfish NNUE SV 1035 vs Stockfish 11:
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile)

+132 elo against Stockfish 11 with Balsa 5 Move Opening Book at 1 min + 0.5 sec TC (1300 games)
+132 elo against Stockfish 11 with SuperGM 4 moves 500 Opening Book at 1 min + 0.5 sec TC (1200 games)

The performance of Stockfish Dev NNUE against Stockfish 11 is +123 elo for now.
https://tests.stockfishchess.org/tests

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Thu Aug 20, 2020 5:31 pm

Test of Cfish NNUE SV 1035

Program Elo + - Games Score Av.Op. Draws

1 Cfish NNUE SV 1035 (Hybrid) : 2471 12 11 1421 69.2 % 2329 55.7 %
2 Stockfish 11 x64 bmi2 : 2329 11 12 1421 30.8 % 2471 55.7 %

Individual statistics:

1 Cfish NNUE SV 1035 (Hybrid): 2471 1421 (+588,=792,- 41), 69.2 %

Stockfish 11 x64 bmi2 : 1421 (+588,=792,- 41), 69.2 %

2 Stockfish 11 x64 bmi2 : 2329 1421 (+ 41,=792,-588), 30.8 %

Cfish NNUE SV 1035 (Hybrid) : 1421 (+ 41,=792,-588), 30.8 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish 180820 NNUE x64 ELTO BMI2 mingw10 (ChessMan compile /Member of The Outskirt Chess Forum )
http://www.mediafire.com/file/aif2qrkoj ... 6.pgn/file

Great performance (+142 elo) against Stockfish 11 x64 bmi2.
The performance of Cfish NNUE SV 1035 (Hybrid) is +10 elo better than Cfish NNUE SV 1035 (Pure).

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Mon Aug 24, 2020 6:20 pm

10 years Of Chess Engines Test:

For now Cfish is the strongest chess engine. Ivanhoe was the one of the strongest chess engine of 2010

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2576 24 95 200 96.5 % 2000 6.0 %
2 IvanHoe T0.5.4.1 x64 : 2000 95 24 200 3.5 % 2576 6.0 %

Individual statistics:

1 Cfish 190820 x64 bmi2 : 2576 200 (+187,= 12,- 1), 96.5 %

IvanHoe T0.5.4.1 x64 : 200 (+187,= 12,- 1), 96.5 %

2 IvanHoe T0.5.4.1 x64 : 2000 200 (+ 1,= 12,-187), 3.5 %

Cfish 190820 x64 bmi2 : 200 (+ 1,= 12,-187), 3.5 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile/Member of The Outskirt Chess Forum) //SV 20200814-1035 Net
http://www.mediafire.com/file/1irue4f5z ... 7.pgn/file

Elo difference is great ( +576 elo)
The power change of chess programs over the last 10 years is incredible. SV Nets have an important role in the formation of this difference.
Last edited by mehmet123 on Mon Aug 24, 2020 6:48 pm, edited 2 times in total.

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Mon Aug 24, 2020 6:23 pm

10 years Of Chess Engines Test:

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2576 24 95 200 96.5 % 2000 6.0 %
2 IvanHoe T0.5.4.1 x64 : 2000 95 24 200 3.5 % 2576 6.0 %

Individual statistics:

1 Cfish 190820 x64 bmi2 : 2576 200 (+187,= 12,- 1), 96.5 %

IvanHoe T0.5.4.1 x64 : 200 (+187,= 12,- 1), 96.5 %

2 IvanHoe T0.5.4.1 x64 : 2000 200 (+ 1,= 12,-187), 3.5 %

Cfish 190820 x64 bmi2 : 200 (+ 1,= 12,-187), 3.5 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile) //SV 20200814-1035 Net
http://www.mediafire.com/file/6l7k185y9 ... 5.pgn/file

I made another test with 1 min + 0.5 sec TC. The result is same ( +576 elo)

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Mon Aug 24, 2020 6:31 pm

Cfish lost 2 out of 400 games against Ivanhoe. 2 lost games are the same Sicilian Variant.


Ivanhoe performed very well in this variant beyond its strength although it was too weak for Cfish.
I made a new match with all games starting with this variant.

1. e4 c5 2. Nc3 Nc6 3. Nf3 e5 4. Bc4 Be7 5. d3 d6 Opening Match

Program Elo + - Games Score Av.Op. Draws

1 Cfish 190820 x64 bmi2 : 2254 56 54 200 81.2 % 2000 12.5 %
2 IvanHoe T0.5.4.1 x64 : 2000 54 56 200 18.8 % 2254 12.5 %

Score of Cfish 190820 x64 bmi2 vs IvanHoe T0.5.4.1 x64 Game_Mode: 150 - 25 - 25 [0.813]
... Cfish 190820 x64 bmi2 playing White: 57 - 25 - 18 [0.660] 100
... Cfish 190820 x64 bmi2 playing Black: 93 - 0 - 7 [0.965] 100
Elo difference: 254.7 +/- 55.9, LOS: 100.0 %, DrawRatio: 12.5 %

Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Cfish NNUE 190820 x64 BMI2 mingw10 (ChessMan compile) //SV 20200814-1035 Net
http://www.mediafire.com/file/flen98ckm ... 6.pgn/file

Ivanhoe won 25 games with black at this variant. But Ivanhoe never managed to win games with white at this variant.
The performance of Cfish with whites in this variant is well below expectations.(% 66)
But it's performance with black in this variant is very well (% 96.5)

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 25, 2020 1:54 pm

Cfish vs Stockfish Match:

Program Elo + - Games Score Av.Op. Draws

1 Cfish 230820 x64 bmi2 : 2405 9 8 1000 51.5 % 2395 84.3 %
2 Stockfish 240820 x64 bmi2 : 2395 8 9 1000 48.4 % 2405 84.3 %

Individual statistics:

1 Cfish 230820 x64 bmi2: 2405 1000 (+ 94,=843,- 63), 51.5 %

Stockfish 240820 x64 bmi2 : 1000 (+ 94,=843,- 63), 51.5 %

2 Stockfish 240820 x64 bmi2 : 2395 1000 (+ 63,=843,- 94), 48.5 %

Cfish 230820 x64 bmi2 : 1000 (+ 63,=843,- 94), 48.4 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation1: Cfish 230820 x64 BMI2 mingw 10.2 (ChessMan compile) // Default Net
Compilation2: Stockfish 240820 x64 bmi2 (Official compile)// Default Net
http://www.mediafire.com/file/gjjn3dfeq ... 0.pgn/file

Cfish is a bit behind in the code update but managed to beat Stockfish.
+10 elo difference over Stockfish is great success.

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 25, 2020 4:43 pm

Test of SV 20200824-1705

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1035 : 2423 12 12 3000 56.5 % 2377 3.0 %
2 Stockfish NNUE SV 1705 : 2377 12 12 3000 43.5 % 2423 3.0 %

Individual statistics:

1 Stockfish NNUE SV 1035 : 2423 3000 (+1650,= 90,-1260), 56.5 %

Stockfish NNUE SV 1705 : 3000 (+1650,= 90,-1260), 56.5 %

2 Stockfish NNUE SV 1705 : 2377 3000 (+1260,= 90,-1650), 43.5 %

Stockfish NNUE SV 1035 : 3000 (+1260,= 90,-1650), 43.5 %

Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 10 kn/move, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 240820 BMI2 mingw 10.2(ChessMan compile)
http://www.mediafire.com/file/4ci326l9j ... 1.pgn/file

At 10 kn/move match SV 1705 net is very weaker ( - 46 elo) than SV 1035 net.

mehmet123
Posts: 201
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 25, 2020 4:51 pm

Test of SV 20200824-1705

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1035 : 2407 10 10 3000 52.1 % 2393 35.8 %
2 Stockfish NNUE SV 1705 : 2393 10 10 3000 47.9 % 2407 35.8 %

Individual statistics:

1 Stockfish NNUE SV 1035 : 2407 3000 (+1026,=1074,-900), 52.1 %

Stockfish NNUE SV 1705 : 3000 (+1026,=1074,-900), 52.1 %

2 Stockfish NNUE SV 1705 : 2393 3000 (+900,=1074,-1026), 47.9 %

Stockfish NNUE SV 1035 : 3000 (+900,=1074,-1026), 47.9 %


Game Conditions: Cutechess Gui, 1 Core ( i7 9750h), 20 kn/move, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 240820 BMI2 mingw 10.2(ChessMan compile)
http://www.mediafire.com/file/jvca3w830 ... 2.pgn/file

At 20 kn/move match the performance of SV 1705 net is much better ( -14 elo) according to its 10 kn/ move match performance.

Post Reply