TalkChess.com

mehmet123 wrote: ↑Fri Jul 31, 2020 9:30 pm At this time control (30 sec+ 0.5 ) all latest Stockfish NNUE versions easily passed +100 elo barrier against Stockfish 11 dev.
At 1 min + 0.5 sec time control all latest Stockfish NNUE versions easily passed +90 elo barrier against Stockfish 11 dev.
But in my tests the performance of latest Stockfish NNUE versions against Stockfish 11 dev at 6 cores (2 min + 1 sec) are between 50-60 elo.

Stockfish NNUE has 2 main problems.
It doesn't scale with increasing time control and it doesn't have an aggressive play style against weak engines.

The first problem could simply be elo compression.
Test SFdev vs SF11 too.

An interesting challenge would be to integrate some sort of working contempt into NNUE, not just for playing but analysis purposes, too.

Test of SV 20200729-2214 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2214 : 2426 28 27 162 57.4 % 2374 72.8 %
2 Stockfish 170720 x64 bmi2 : 2374 27 28 162 42.6 % 2426 72.8 %

Individual statistics:

1 Stockfish NNUE SV 2214 : 2426 162 (+ 34,=118,- 10), 57.4 %

Stockfish 170720 x64 bmi2 : 162 (+ 34,=118,- 10), 57.4 %

2 Stockfish 170720 x64 bmi2 : 2374 162 (+ 10,=118,- 34), 42.6 %

Stockfish NNUE SV 2214 : 162 (+ 10,=118,- 34), 42.6 %

Arena Gui, 6 Core (i7 9750h), 2 min + 1 sec TC, Balsa 5 Move Opening Book,512 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10

http://www.mediafire.com/file/5ctt3cah0 ... 0.pgn/file

Test of 20200729-1743 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1743 : 2437 36 32 100 60.5 % 2363 73.0 %
2 Stockfish 170720 x64 bmi2 : 2363 32 36 100 39.5 % 2437 73.0 %

Individual statistics:

1 Stockfish NNUE SV 1743 : 2437 100 (+ 24,= 73,- 3), 60.5 %

Stockfish 170720 x64 bmi2 : 100 (+ 24,= 73,- 3), 60.5 %

2 Stockfish 170720 x64 bmi2 : 2363 100 (+ 3,= 73,- 24), 39.5 %

Stockfish NNUE SV 1743 : 100 (+ 3,= 73,- 24), 39.5 %

Arena Gui, 6 Core (i7 9750h), 2 min + 1 sec TC, Balsa 5 Move Opening Book, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10 (From "ChessMan"/The Outskirts Chess Forum Member)
http://www.mediafire.com/file/9hr160ffb ... 0.pgn/file

Great record (+ 74 elo) at these conditions (6 core, 2 min + 1 sec).

Test of 20200801-1209 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SF 1209 : 2454 25 24 300 65.0 % 23.46 58.7 %
2 Stockfish 170720 x64 bmi2 :2346 24 25 300 35.0 % 2454 58.7 %

Individual statistics:

1 Stockfish NNUE SF 1209 : 2454 300 (+107,=176,- 17), 65.0 %

Stockfish 170720 x64 bmi2 : 300 (+107,=176,- 17), 65.0 %

2 Stockfish 170720 x64 bmi2 : 2346 300 (+ 17,=176,-107), 35.0 %

Stockfish NNUE SF 1209 : 300 (+ 17,=176,-107), 35.0 %

Game conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10

http://www.mediafire.com/file/eswm757po ... 2.pgn/file

Test of 0200801-1515 Net:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1515 : 2453 30 28 200 64.8 % 23.47 61.5 %
2 Stockfish 170720 x64 bmi2 : 2347 28 30 200 35.2 % 2453 61.5 %


Individual statistics:

1 Stockfish NNUE SV 1515 : 2453 200 (+ 68,=123,- 9), 64.8 %

Stockfish 170720 x64 bmi2 : 200 (+ 68,=123,- 9), 64.8 %

2 Stockfish 170720 x64 bmi2 : 2347 200 (+ 9,=123,- 68), 35.2 %

Stockfish NNUE SV 1515 : 200 (+ 9,=123,- 68), 35.2 %

Game conditions: Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10

http://www.mediafire.com/file/iyiec4g3w ... 2.pgn/file


The difference of over 100 elo against Stockfish dev (30 sec +0.5 sec TC) is now ordinary. Computer chess world has entered an interesting period.
Chess engines using networks will increase much more soon.

No Opening Book Test

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1515 : 2457 22 22 401 65.8 % 2343 56.4 %
2 Stockfish 170720 x64 bmi2 :2343 22 22 401 34.2 % 2457 56.4 %

Individual statistics:

1 Stockfish NNUE SV 1515 : 2457 401 (+151,=226,- 24), 65.8 %

Stockfish 170720 x64 bmi2 : 401 (+151,=226,- 24), 65.8 %

2 Stockfish 170720 x64 bmi2 : 2343 401 (+ 24,=226,-151), 34.2 %

Stockfish NNUE SV 1515 : 401 (+ 24,=226,-151), 34.2 %

Game Conditions:Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, No opening book, Ponder off, 16 Mb Hash
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/ixwfpid3w ... 4.pgn/file

Stockfish NNUE SV 1515 against Stockfish 170720: + 106 elo (Balsa 5 move opening book)
Stockfish NNUE SV 1515 against Stockfish 170720: + 114 elo (No opening book)

Test of 0200801-1515 Net:

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 1515 : 2446 21 20 400 62.9 % 2354 61.8 %
2 Stockfish 170720 x64 bmi2 : 2354 20 21 400 37.1 % 2446 61.8 %

Individual statistics:

1 Stockfish NNUE SV 1515 : 2446 400 (+128,=247,- 25), 62.9 %

Stockfish 170720 x64 bmi2 : 400 (+128,=247,- 25), 62.9 %

2 Stockfish 170720 x64 bmi2 : 2354 400 (+ 25,=247,-128), 37.1 %

Stockfish NNUE SV 1515 : 400 (+ 25,=247,-128), 37.1 %

Game conditions: Cutechess Gui, 1 Core (i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Move Opening Book, 64 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/z6hirj62t ... 6.pgn/file

Good performance (+ 92 elo) at this time condition ( 2min + 0.5 sec)

Test of 20200802-2257 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2257 : 2443 21 20 340 62.1 % 2357 65.9 %
2 Stockfish 170720 x64 bmi2 : 2357 20 21 340 37.9 % 2443 65.9 %

Individual statistics:

1 Stockfish NNUE SV 2257 : 2443 340 (+ 99,=224,- 17), 62.1 %

Stockfish 170720 x64 bmi2 : 340 (+ 99,=224,- 17), 62.1 %

2 Stockfish 170720 x64 bmi2 : 2357 340 (+ 17,=224,- 99), 37.9 %

Stockfish NNUE SV 2257 : 340 (+ 17,=224,- 99), 37.9 %


Game conditions: Cutechess Gui, 1 Core (i7 9750h), 2 min + 0.5 sec TC, Balsa 5 Move Opening Book, 128 Mb Hash, Ponder Off
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10
http://www.mediafire.com/file/u88jx1yke ... 1.pgn/file

SV 2257 (+86 elo) failed to pass SV 1515 (+92 elo) at this time condition (2 min + 0.5 sec).
But the number of games aren't enough to say something with certainty.

Test of 20200803-2108 Net

Program Elo + - Games Score Av.Op. Draws

1 Stockfish NNUE SV 2108 : 2464 23 22 400 67.6 % 2336 53.8 %
2 Stockfish 170720 x64 bmi2 : 2336 22 23 400 32.4 % 2464 53.8 %

Individual statistics:

1 Stockfish NNUE SV 2108 : 2464 400 (+163,=215,- 22), 67.6 %

Stockfish 170720 x64 bmi2 : 400 (+163,=215,- 22), 67.6 %

2 Stockfish 170720 x64 bmi2 : 2336 400 (+ 22,=215,-163), 32.4 %

Stockfish NNUE SV 2108 : 400 (+ 22,=215,-163), 32.4 %

Game Conditions:Cutechess Gui, 1 Core (i7 9750h), 30 sec + 0.5 sec TC, No opening book, Ponder off, 64 Mb Hash
Compilation: Stockfish NNUE 250720 x64 Haswell BMI2 256 mingw10 (From "ChessMan"/The Outskirts Chess Forum Member)
http://www.mediafire.com/file/pja5fhgii ... 0.pgn/file

Great record ( +128 elo) at this time condition (30 sec + 0.5 sec )