Stockfish NNUE SV Tests

[mehmet123]

Great Chess Engine, Great Nets and Great Compilation.
Thanks to Stockfish Developers, Henk Drost, Sergio Vieri and "ChessMan ".

AlphaZero, Lc0 and now Stockfish NNUE. Nothing will be the same anymore in computer chess.
We can see network usage in many engines soon. I hope to see Komodo NNUE chess engine very soon

[mehmet123]

Stockfish NNUE has a real scale problem. It doesn 't scale with time increment. I have mentioned this problem before in some chess forums.
At 1 core, 30 sec + 0.5 sec time control the record is +128 elo in my tests but at 6 cores, 2 min + 1 sec tc the record is +74 elo.
At Mark Young' s last test (32 cores , 30 min + 30 sec ) for now elo difference is only +14 elo (after 73 games ).
Stockfish NNUE needs more quality nets for fhe solution of this problem. I am sure that Stockfish NNUE developer team will overcome this problem.

[Gary Internet]

Stockfish NNUE has a real scale problem. It doesn 't scale with time increment. I have mentioned this problem before in some chess forums.
At 1 core, 30 sec + 0.5 sec time control the record is +128 elo in my tests but at 6 cores, 2 min + 1 sec tc the record is +74 elo.
At Mark Young' s last test (32 cores , 30 min + 30 sec ) for now elo difference is only +14 elo (after 73 games ).
Stockfish NNUE needs more quality nets for fhe solution of this problem. I am sure that Stockfish NNUE developer team will overcome this problem.

TCEC Superfinal Conditions of 176 threads, 64 GB hash and a time control of 90' + 10" would reduce this gap to almost nothing. The gap would be small enough that submitting SF NNUE instead of standard SF might not actually change much in terms of performance against Lc0 or Allie.

[mehmet123]

Test of 20200803-2108 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2108 : 2446 23 22 350 63.0 % 2354 60.9 %
2 Stockfish 170720 x64 bmi2 : 2354 22 23 350 37.0 % 2446 60.9 %

Individual statistics:

1 Stockfish NNUE SV 2108 : 2446 350 (+114,=213,- 23), 63.0 %

Stockfish 170720 x64 bmi2 : 350 (+114,=213,- 23), 63.0 %

2 Stockfish 170720 x64 bmi2 : 2354 350 (+ 23,=213,-114), 37.0 %

Stockfish NNUE SV 2108 : 350 (+ 23,=213,-114), 37.0 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/ms2ss9zmh ... 2.pgn/file

[mehmet123]

Test of qh-fine-0803-5 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish qh-fine-0803-5 : 2453 21 20 400 64.8 % 23.47 60.5 %
2 Stockfish 170720 x64 bmi2 :2347 20 21 400 35.2 % 2453 60.5 %

Individual statistics:

1 Stockfish qh-fine-0803-5 : 2453 400 (+138,=242,- 20), 64.8 %

Stockfish 170720 x64 bmi2 : 400 (+138,=242,- 20), 64.8 %

2 Stockfish 170720 x64 bmi2 : 2347 400 (+ 20,=242,-138), 35.2 %

Stockfish qh-fine-0803-5 : 400 (+ 20,=242,-138), 35.2 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/xcgq7jgew ... 3.pgn/file

This net come from an another source than SV nets. This net trained on reinforcement learn and tuned using CCRL games.
It's performance is really great ( +106 elo) at this time control ( 1 min + 0.5 sec)

[mehmet123]

Test of qh-fine-0803-5 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish qh-fine-0803-5 : 2443 25 23 200 62.3 % 2357 72.5 %
2 Stockfish 170720 x64 bmi2 : 2357 23 25 200 37.8 % 2443 72.5 %

Individual statistics:

1 Stockfish qh-fine-0803-5 : 2443 200 (+ 52,=145,- 3), 62.2 %

Stockfish 170720 x64 bmi2 : 200 (+ 52,=145,- 3), 62.3 %

2 Stockfish 170720 x64 bmi2 : 2357 200 (+ 3,=145,- 52), 37.8 %

Stockfish qh-fine-0803-5 : 200 (+ 3,=145,- 52), 37.8 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 5 min + 0.2 sec TC, Balsa 5 Move Opening Book, 512 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/kt58hqnab ... 6.pgn/file

Another great performance of (+ 86 elo) qh-fine-0803-5 net at this time control (5 min + 0.2 sec)

[mehmet123]

In my 2 tests the performance of Stockfish NNUE qh-fine-0803-5 is great.

Download link of qh-fine-0803-5 net:
https://www.dropbox.com/s/71qy3j8gtgch2 ... .zip?dl=00

[fastgm]

The qh-fine-0803-5 net is Sergio Vieris net 20200729-0109
http://talkchess.com/forum3/viewtopic.p ... 72#p854719

[mehmet123]

Sad news. Thanks for your information.
The only good thing about this news is that i shared this network in the right topic (Stockfish NNUE SV Tests)

[mehmet123]

Test of 20200804-1817 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1817 : 2452 18 18 608 64.6 % 2348 56.1 %
2 Stockfish 170720 x64 bmi2 : 2348 18 18 608 35.4 % 2452 56.1 %

Individual statistics:

1 Stockfish NNUE SV 1817 : 2452 608 (+222,=341,- 45), 64.6 %

Stockfish 170720 x64 bmi2 : 608 (+222,=341,- 45), 64.6 %

2 Stockfish 170720 x64 bmi2 : 2348 608 (+ 45,=341,-222), 35.4 %

Stockfish NNUE SV 1817 : 608 (+ 45,=341,-222), 35.4 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/7c58yheca ... 8.pgn/file

Another good performance (+104 elo) at this time control (1 min + 0.5 sec)