Stockfish NNUE SV Tests

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Sun Jul 26, 2020 10:50 am

SV 0511 Net vs SV 2344 Net

1 core, 30 sec +0.5 sec test
SV 0511: 78 elo stronger than Stockfish 170720
SV 2344: 84 elo stronger than Stockfish 170720

6 core, 2 min + 1 sec test
SV 0511: 52 elo stronger than Stockfish 170720
SV 2344: 48 elo stronger than Stockfish 170720

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Mon Jul 27, 2020 9:48 pm

Test of SV 20200727-2151 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2151 : 2446 23 22 360 62.9 % 2354 58.6 %
2 Stockfish 170720 x64 bmi2 : 2354 22 23 360 37.1 % 2446 58.6 %

Individual statistics:

1 Stockfish NNUE SV 2151 : 2446 360 (+121,=211,- 28), 62.9 %

Stockfish 170720 x64 bmi2 : 360 (+121,=211,- 28), 62.9 %

2 Stockfish 170720 x64 bmi2 : 2354 360 (+ 28,=211,-121), 37.1 %

Stockfish NNUE SV 2151 : 360 (+ 28,=211,-121), 37.1 %


Game conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10 (From "ChessMan" /The Outskirts Chess Forum Member)
http://www.mediafire.com/file/u66v5i42j ... 0.pgn/file

92 elo difference is a new record at this time condition.

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 28, 2020 2:46 am

Test of SV 20200727-2151 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2151 : 2440 20 19 440 61.3 % 2360 63.0 %
2 Stockfish 170720 x64 bmi2 : 2360 19 20 440 38.8 % 2440 63.0 %

Individual statistics:

1 Stockfish NNUE SV 2151 : 2440 440 (+131,=277,- 32), 61.2 %

Stockfish 170720 x64 bmi2 : 440 (+131,=277,- 32), 61.3 %

2 Stockfish 170720 x64 bmi2 : 2360 440 (+ 32,=277,-131), 38.8 %

Stockfish NNUE SV 2151 : 440 (+ 32,=277,-131), 38.8 %

Game conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10 (From "ChessMan"/The Outskirts Chess Forum Member)
http://www.mediafire.com/file/d9xj3kv6p ... 4.pgn/file

Another great result. +80 elo difference is another record at this time condition.
For compare the performance of Stockfish NNUE SV 2141 against Stockfish 170720 x64 bmi2 is +52 elo at this time condition.

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Tue Jul 28, 2020 7:21 pm

Test of 20200728-1442 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish SV 1442 : 2460 26 25 300 66.7 % 2340 56.0 %
2 Stockfish 170720 x64 bmi2 : 2340 25 26 300 33.3 % 2460 56.0 %

Individual statistics:

1 Stockfish SV 1442 : 2460 300 (+116,=168,- 16), 66.7 %

Stockfish 170720 x64 bmi2 : 300 (+116,=168,- 16), 66.7 %

2 Stockfish 170720 x64 bmi2 : 2340 300 (+ 16,=168,-116), 33.3 %

Stockfish SV 1442 : 300 (+ 16,=168,-116), 33.3 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/jyjokr90b ... 7.pgn/file


Incredible result. +120 elo difference is a new record at this time condition.

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Jul 29, 2020 2:38 am

Test of 20200728-1442 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish SV 1442 : 2446 21 20 360 63.1 % 2354 63.9 %
2 Stockfish 170720 x64 bmi2 : 2354 20 21 360 36.9 % 2446 63.9 %

Individual statistics:

1 Stockfish SV 1442 : 2446 360 (+112,=230,- 18), 63.1 %

Stockfish 170720 x64 bmi2 : 360 (+112,=230,- 18), 63.1 %

2 Stockfish 170720 x64 bmi2 : 2354 360 (+ 18,=230,-112), 36.9 %

Stockfish SV 1442 : 360 (+ 18,=230,-112), 36.9 %


Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/pk8xw9gh3 ... 8.pgn/file

+92 elo difference , it's a new record at this time condition (1 min + 0.5 sec)

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Jul 29, 2020 7:20 pm

Test of SV 20200729-2214 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2214 : 2456 31 29 200 65.5 % 23.44 59.0 %
2 Stockfish 170720 x64 bmi2 : 2344 29 31 200 34.5 % 2456 59.0 %

Individual statistics:

1 Stockfish NNUE SV 2214 : 2456 200 (+ 72,=118,- 10), 65.5 %

Stockfish 170720 x64 bmi2 : 200 (+ 72,=118,- 10), 65.5 %

2 Stockfish 170720 x64 bmi2 : 2344 200 (+ 10,=118,- 72), 34.5 %

Stockfish NNUE SV 2214 : 200 (+ 10,=118,- 72), 34.5 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10 (From "ChessMan"/The Outskirts Chess Forum Member)
http://www.mediafire.com/file/fs5tqcfkp ... 0.pgn/file

+112 elo difference is a great record at this time condition ( 1 min +0.5 sec).
Previous records at this time condition were +92 elo (20200728-1442), +80 elo (20200727-2151),

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Wed Jul 29, 2020 9:53 pm

Stockfish NNUE SV 2214 finds the legendary move Rxd4 (Kasparov-Topalov /1999) lesser than 1 second with 1 core (Core i7 -9750h).
This is very impressive. Stockfish 170720 x64 bmi2 finds this move nearly 1 minute with 6 cores (Core i7 -9750h)
http://www.mediafire.com/view/jb82md51p ... 9.PNG/file

An article about this game:
https://www.chess.com/blog/SamCopeland/ ... palov-1999

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Thu Jul 30, 2020 10:23 pm

Test of SV 20200731-0111 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 111 : 2448 25 25 300 63.5 % 2352 57.7 %
2 Stockfish 170720 x64 bmi2 : 2352 25 25 300 36.5 % 2448 57.7 %

Individual statistics:

1 Stockfish NNUE SV 111 : 2448 300 (+104,=173,- 23), 63.5 %

Stockfish 170720 x64 bmi2 : 300 (+104,=173,- 23), 63.5 %

2 Stockfish 170720 x64 bmi2 : 2352 300 (+ 23,=173,-104), 36.5 %

Stockfish NNUE SV 111 : 300 (+ 23,=173,-104), 36.5 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 1 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/gknvb7z3m ... 1.pgn/file

Not record but a good a performance (+96 elo).

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Fri Jul 31, 2020 7:12 pm

Test of SV 20200731-0252 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 0252 : 2456 22 21 400 65.5 % 2344 58.5 %
2 Stockfish 170720 x64 bmi2 : 2344 21 22 400 34.5 % 2456 58.5 %

Individual statistics:

1 Stockfish NNUE SV 0252 : 2456 400 (+145,=234,- 21), 65.5 %

Stockfish 170720 x64 bmi2 : 400 (+145,=234,- 21), 65.5 %

2 Stockfish 170720 x64 bmi2 : 2344 400 (+ 21,=234,-145), 34.5 %

Stockfish NNUE SV 0252 : 400 (+ 21,=234,-145), 34.5 %

Game Conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/81986pgn3 ... 4.pgn/file

Another great performance (+112 elo)

mehmet123
Posts: 254
Joined: Sun Jan 26, 2020 9:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: Stockfish NNUE SV Tests

Post by mehmet123 » Fri Jul 31, 2020 7:30 pm

At this time control (30 sec+ 0.5 ) all latest Stockfish NNUE versions easily passed +100 elo barrier against Stockfish 11 dev.
At 1 min + 0.5 sec time control all latest Stockfish NNUE versions easily passed +90 elo barrier against Stockfish 11 dev.
But in my tests the performance of latest Stockfish NNUE versions against Stockfish 11 dev at 6 cores (2 min + 1 sec) are between 50-60 elo.

Stockfish NNUE has 2 main problems.
It doesn't scale with increasing time control and it doesn't have an aggressive play style against weak engines.

Post Reply