A match between SF12+NNUE and Leele ver.0.26.2

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
corres
Posts: 3657
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

A match between SF12+NNUE and Leele ver.0.26.2

Post by corres » Tue Sep 08, 2020 8:06 pm

My system is:
OS Windows 10 x64 bits
Processor Ryzen 9 3950x 16 x 4.0 GHz fixed, no power-boost, no SMT
Ram 32 GB
TRIAD = 1 x RTX 2080 Ti OC + 2 x RTX 2060 OC GPUs (chess power ~ 2 x RTX 2080 Ti)
Chess GUI Fritz 14
Other params:
SF12 +NNUE exe is optimized for avx2
Hash for CPU 2 GB
Net for NNUE nn-82215d0fd0df.nnue (the default)
Leela version is for CUDA 11 and cudnn 7.6.4
Leela parameters were "Kiudee" (Default) and Threads = 6 x 4.0 GHz, NNCachesize = 20000000,
Backend = Multiplexing
Net for Leela SV-384x30-T60-3010
Endgame databases 6 men syzygy and 5 men Nalimov for chess GUI
Opening Book My Mini-book with 100 opening position with alternated color.
Tournament time 1 min + 2 sec / move
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
Note
Leela speed with CUDA11 and cudnn 7.6.4 is faster about 30 % as her speed with CUDA 10 and cudnn 7.4.2
(the default one) on SV-384x30-T60-3010 net

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by mwyoung » Tue Sep 08, 2020 9:53 pm

corres wrote:
Tue Sep 08, 2020 8:06 pm
My system is:
OS Windows 10 x64 bits
Processor Ryzen 9 3950x 16 x 4.0 GHz fixed, no power-boost, no SMT
Ram 32 GB
TRIAD = 1 x RTX 2080 Ti OC + 2 x RTX 2060 OC GPUs (chess power ~ 2 x RTX 2080 Ti)
Chess GUI Fritz 14
Other params:
SF12 +NNUE exe is optimized for avx2
Hash for CPU 2 GB
Net for NNUE nn-82215d0fd0df.nnue (the default)
Leela version is for CUDA 11 and cudnn 7.6.4
Leela parameters were "Kiudee" (Default) and Threads = 6 x 4.0 GHz, NNCachesize = 20000000,
Backend = Multiplexing
Net for Leela SV-384x30-T60-3010
Endgame databases 6 men syzygy and 5 men Nalimov for chess GUI
Opening Book My Mini-book with 100 opening position with alternated color.
Tournament time 1 min + 2 sec / move
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
Note
Leela speed with CUDA11 and cudnn 7.6.4 is faster about 30 % as her speed with CUDA 10 and cudnn 7.4.2
(the default one) on SV-384x30-T60-3010 net
Very very interesting...
What out for the SF fanboys. They are vicious . :lol:

Code: Select all

Wins	Losses	Draws
9
6
185
Points		Games
101.5
/	
200
Winning percentage
50.75

Elo difference : 
+5
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

corres
Posts: 3657
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by corres » Tue Sep 08, 2020 10:04 pm

Maybe the developers of Leela should modify the the used CUDA and cudnn .dll.
+30 % speed gives about 10-20 Elo.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by mwyoung » Tue Sep 08, 2020 10:13 pm

corres wrote:
Tue Sep 08, 2020 10:04 pm
Maybe the developers of Leela should modify the the used CUDA and cudnn .dll.
+30 % speed gives about 10-20 Elo.
I am unclear on the Elo gain here. My 200 game test with SF12+NNUE and Leele ver.0.26.2.

SF 12 won by +24 Elo using only 1 RTX 2080ti vs 32 thread SF 12 on a 2950x.

But I was using NN J92-115 for Lc0.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

Scipion
Posts: 8
Joined: Sat Jun 13, 2020 1:17 pm
Full name: Al Stemms

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by Scipion » Tue Sep 08, 2020 10:28 pm

Could you please post a SF12 lost game ?
Never seen one yet in all my engines tournaments.

OliverBr
Posts: 669
Joined: Tue Dec 18, 2007 8:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch
Contact:

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by OliverBr » Tue Sep 08, 2020 10:29 pm

corres wrote:
Tue Sep 08, 2020 8:06 pm
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
This is no result at all, you need at least 5000 games in order to announce a result.
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by mwyoung » Tue Sep 08, 2020 10:31 pm

OliverBr wrote:
Tue Sep 08, 2020 10:29 pm
corres wrote:
Tue Sep 08, 2020 8:06 pm
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
This is no result at all, you need at least 5000 games in order to announce a result.
True unless the games are played by TCEC. :lol:
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

corres
Posts: 3657
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by corres » Tue Sep 08, 2020 10:36 pm

OliverBr wrote:
Tue Sep 08, 2020 10:29 pm
corres wrote:
Tue Sep 08, 2020 8:06 pm
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
This is no result at all, you need at least 5000 games in order to announce a result.
Sorry, Oliver, I am not an official tester, I make tests for my interest only.
I report the parameters of my test, and others can level my test as they want.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by mwyoung » Tue Sep 08, 2020 10:52 pm

corres wrote:
Tue Sep 08, 2020 10:36 pm
OliverBr wrote:
Tue Sep 08, 2020 10:29 pm
corres wrote:
Tue Sep 08, 2020 8:06 pm
Result is
SF12+NNUE : kTRIAD = 9 : 6 (185 draw) 200 games
This is no result at all, you need at least 5000 games in order to announce a result.
Sorry, Oliver, I am not an official tester, I make tests for my interest only.
I report the parameters of my test, and others can level my test as they want.
A tester does not need 5000 games, unless the goal is only to maximize the Elo precision. A useful tool for engine developers.

What are you trying to answer.
1. What is the exact elo deference between A vs B.
2. Who is better only between A vs B, and this usually takes much fewer games to prove statistically.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

OliverBr
Posts: 669
Joined: Tue Dec 18, 2007 8:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch
Contact:

Re: A match between SF12+NNUE and Leele ver.0.26.2

Post by OliverBr » Tue Sep 08, 2020 11:01 pm

mwyoung wrote:
Tue Sep 08, 2020 10:52 pm
What are you trying to answer.
1. What is the exact elo deference between A vs B.
2. Who is better only between A vs B, and this usually takes much fewer games to prove statistically.
Still you need a couple of thousand games.
Believe me, I can do them with a 32-core computer and no, some hundred games are not enough.
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink

Post Reply