A match between SF12+NNUE and Leele ver.0.26.2

[corres]

My system is:
OS Windows 10 x64 bits
Processor Ryzen 9 3950x 16 x 4.0 GHz fixed, no power-boost, no SMT
Ram 32 GB
TRIAD = 1 x RTX 2080 Ti OC + 2 x RTX 2060 OC GPUs (chess power ~ 2 x RTX 2080 Ti)
Chess GUI Fritz 14
Other params:
SF12 +NNUE exe is optimized for avx2
Hash for CPU 2 GB
Net for NNUE nn-82215d0fd0df.nnue (the default)
Leela version is for CUDA 11 and cudnn 7.6.4
Leela parameters were "Kiudee" (Default) and Threads = 6 x 4.0 GHz, NNCachesize = 20000000,
Backend = Multiplexing
Net for Leela SV-384x30-T60-3010
Endgame databases 6 men syzygy and 5 men Nalimov for chess GUI
Opening Book My Mini-book with 100 opening position with alternated color.
Tournament time 1 min + 2 sec / move
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
Note
Leela speed with CUDA11 and cudnn 7.6.4 is faster about 30 % as her speed with CUDA 10 and cudnn 7.4.2
(the default one) on SV-384x30-T60-3010 net

[mwyoung]

My system is:
OS Windows 10 x64 bits
Processor Ryzen 9 3950x 16 x 4.0 GHz fixed, no power-boost, no SMT
Ram 32 GB
TRIAD = 1 x RTX 2080 Ti OC + 2 x RTX 2060 OC GPUs (chess power ~ 2 x RTX 2080 Ti)
Chess GUI Fritz 14
Other params:
SF12 +NNUE exe is optimized for avx2
Hash for CPU 2 GB
Net for NNUE nn-82215d0fd0df.nnue (the default)
Leela version is for CUDA 11 and cudnn 7.6.4
Leela parameters were "Kiudee" (Default) and Threads = 6 x 4.0 GHz, NNCachesize = 20000000,
Backend = Multiplexing
Net for Leela SV-384x30-T60-3010
Endgame databases 6 men syzygy and 5 men Nalimov for chess GUI
Opening Book My Mini-book with 100 opening position with alternated color.
Tournament time 1 min + 2 sec / move
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
Note
Leela speed with CUDA11 and cudnn 7.6.4 is faster about 30 % as her speed with CUDA 10 and cudnn 7.4.2
(the default one) on SV-384x30-T60-3010 net

Very very interesting...
What out for the SF fanboys. They are vicious .

Code: Select all

Wins	Losses	Draws
9
6
185
Points		Games
101.5
/	
200
Winning percentage
50.75

Elo difference : 
+5

[corres]

Maybe the developers of Leela should modify the the used CUDA and cudnn .dll.
+30 % speed gives about 10-20 Elo.

[mwyoung]

Maybe the developers of Leela should modify the the used CUDA and cudnn .dll.
+30 % speed gives about 10-20 Elo.

I am unclear on the Elo gain here. My 200 game test with SF12+NNUE and Leele ver.0.26.2.

SF 12 won by +24 Elo using only 1 RTX 2080ti vs 32 thread SF 12 on a 2950x.

But I was using NN J92-115 for Lc0.

[Scipion]

Could you please post a SF12 lost game ?
Never seen one yet in all my engines tournaments.

[OliverBr]

Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games

This is no result at all, you need at least 5000 games in order to announce a result.

[mwyoung]

Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games

This is no result at all, you need at least 5000 games in order to announce a result.

True unless the games are played by TCEC.

[corres]

Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games

This is no result at all, you need at least 5000 games in order to announce a result.

Sorry, Oliver, I am not an official tester, I make tests for my interest only.
I report the parameters of my test, and others can level my test as they want.

[mwyoung]

Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games

This is no result at all, you need at least 5000 games in order to announce a result.

Sorry, Oliver, I am not an official tester, I make tests for my interest only.
I report the parameters of my test, and others can level my test as they want.

A tester does not need 5000 games, unless the goal is only to maximize the Elo precision. A useful tool for engine developers.

What are you trying to answer.
1. What is the exact elo deference between A vs B.
2. Who is better only between A vs B, and this usually takes much fewer games to prove statistically.

[OliverBr]

What are you trying to answer.
1. What is the exact elo deference between A vs B.
2. Who is better only between A vs B, and this usually takes much fewer games to prove statistically.

Still you need a couple of thousand games.
Believe me, I can do them with a 32-core computer and no, some hundred games are not enough.