Stockfish NNUE SV Tests

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Mon Aug 03, 2020 8:06 pm

Great Chess Engine, Great Nets and Great Compilation.
Thanks to Stockfish Developers, Henk Drost, Sergio Vieri and "ChessMan ".

AlphaZero, Lc0 and now Stockfish NNUE. Nothing will be the same anymore in computer chess.
We can see network usage in many engines soon. I hope to see Komodo NNUE chess engine very soon

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 7:07 am

Stockfish NNUE has a real scale problem. It doesn 't scale with time increment. I have mentioned this problem before in some chess forums.
At 1 core, 30 sec + 0.5 sec time control the record is +128 elo in my tests but at 6 cores, 2 min + 1 sec tc the record is +74 elo.
At Mark Young' s last test (32 cores , 30 min + 30 sec ) for now elo difference is only +14 elo (after 73 games ).
Stockfish NNUE needs more quality nets for fhe solution of this problem. I am sure that Stockfish NNUE developer team will overcome this problem.

Gary Internet
Posts: 60
Joined: Thu Jan 04, 2018 6:09 pm

Re: Stockfish NNUE SV Tests

Post by Gary Internet » Tue Aug 04, 2020 10:57 am

mehmet123 wrote:
Tue Aug 04, 2020 7:07 am
Stockfish NNUE has a real scale problem. It doesn 't scale with time increment. I have mentioned this problem before in some chess forums.
At 1 core, 30 sec + 0.5 sec time control the record is +128 elo in my tests but at 6 cores, 2 min + 1 sec tc the record is +74 elo.
At Mark Young' s last test (32 cores , 30 min + 30 sec ) for now elo difference is only +14 elo (after 73 games ).
Stockfish NNUE needs more quality nets for fhe solution of this problem. I am sure that Stockfish NNUE developer team will overcome this problem.
TCEC Superfinal Conditions of 176 threads, 64 GB hash and a time control of 90' + 10" would reduce this gap to almost nothing. The gap would be small enough that submitting SF NNUE instead of standard SF might not actually change much in terms of performance against Lc0 or Allie.

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 4:20 pm

Test of 20200803-2108 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2108 : 2446 23 22 350 63.0 % 2354 60.9 %
2 Stockfish 170720 x64 bmi2 : 2354 22 23 350 37.0 % 2446 60.9 %

Individual statistics:

1 Stockfish NNUE SV 2108 : 2446 350 (+114,=213,- 23), 63.0 %

Stockfish 170720 x64 bmi2 : 350 (+114,=213,- 23), 63.0 %

2 Stockfish 170720 x64 bmi2 : 2354 350 (+ 23,=213,-114), 37.0 %

Stockfish NNUE SV 2108 : 350 (+ 23,=213,-114), 37.0 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/ms2ss9zmh ... 2.pgn/file

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 4:32 pm

Test of qh-fine-0803-5 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish qh-fine-0803-5 : 2453 21 20 400 64.8 % 23.47 60.5 %
2 Stockfish 170720 x64 bmi2 :2347 20 21 400 35.2 % 2453 60.5 %

Individual statistics:

1 Stockfish qh-fine-0803-5 : 2453 400 (+138,=242,- 20), 64.8 %

Stockfish 170720 x64 bmi2 : 400 (+138,=242,- 20), 64.8 %

2 Stockfish 170720 x64 bmi2 : 2347 400 (+ 20,=242,-138), 35.2 %

Stockfish qh-fine-0803-5 : 400 (+ 20,=242,-138), 35.2 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/xcgq7jgew ... 3.pgn/file

This net come from an another source than SV nets. This net trained on reinforcement learn and tuned using CCRL games.
It's performance is really great ( +106 elo) at this time control ( 1 min + 0.5 sec)

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 5:55 pm

Test of qh-fine-0803-5 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish qh-fine-0803-5 : 2443 25 23 200 62.3 % 2357 72.5 %
2 Stockfish 170720 x64 bmi2 : 2357 23 25 200 37.8 % 2443 72.5 %

Individual statistics:

1 Stockfish qh-fine-0803-5 : 2443 200 (+ 52,=145,- 3), 62.2 %

Stockfish 170720 x64 bmi2 : 200 (+ 52,=145,- 3), 62.3 %

2 Stockfish 170720 x64 bmi2 : 2357 200 (+ 3,=145,- 52), 37.8 %

Stockfish qh-fine-0803-5 : 200 (+ 3,=145,- 52), 37.8 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 5 min + 0.2 sec TC, Balsa 5 Move Opening Book, 512 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/kt58hqnab ... 6.pgn/file

Another great performance of (+ 86 elo) qh-fine-0803-5 net at this time control (5 min + 0.2 sec)

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 6:02 pm

In my 2 tests the performance of Stockfish NNUE qh-fine-0803-5 is great.

Download link of qh-fine-0803-5 net:
https://www.dropbox.com/s/71qy3j8gtgch2 ... .zip?dl=00

fastgm
Posts: 604
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: Stockfish NNUE SV Tests

Post by fastgm » Tue Aug 04, 2020 6:31 pm

The qh-fine-0803-5 net is Sergio Vieris net 20200729-0109
viewtopic.php?f=2&t=74672#p854719

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 6:51 pm

Sad news. Thanks for your information.
The only good thing about this news is that i shared this network in the right topic (Stockfish NNUE SV Tests) :)

mehmet123
Posts: 223
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Aug 04, 2020 9:28 pm

Test of 20200804-1817 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1817 : 2452 18 18 608 64.6 % 2348 56.1 %
2 Stockfish 170720 x64 bmi2 : 2348 18 18 608 35.4 % 2452 56.1 %

Individual statistics:

1 Stockfish NNUE SV 1817 : 2452 608 (+222,=341,- 45), 64.6 %

Stockfish 170720 x64 bmi2 : 608 (+222,=341,- 45), 64.6 %

2 Stockfish 170720 x64 bmi2 : 2348 608 (+ 45,=341,-222), 35.4 %

Stockfish NNUE SV 1817 : 608 (+ 45,=341,-222), 35.4 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/7c58yheca ... 8.pgn/file

Another good performance (+104 elo) at this time control (1 min + 0.5 sec)

Post Reply