LCZero: Progress and Scaling. Relation to CCRL Elo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Milos » Tue May 08, 2018 11:35 am

mirek wrote:
Tue May 08, 2018 9:21 am
I wonder if it's just an accident that the regression started around the time v8 was released?
I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7.
Last edited by Milos on Tue May 08, 2018 1:16 pm, edited 3 times in total.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Milos » Tue May 08, 2018 11:35 am

duplicate
Last edited by Milos on Tue May 08, 2018 1:27 pm, edited 3 times in total.

mirek
Posts: 42
Joined: Sat Mar 24, 2018 3:18 pm

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by mirek » Tue May 08, 2018 12:22 pm

Milos wrote:
Tue May 08, 2018 11:35 am
I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7
There could also possibly be a bug that shows only in in training though.
About cuDNN yes my results are also similar in OCL vs. cuDNN lc0 but when testing against different engines the cuDNN version had only like 20, 30 elo edge, basically still within the 1 sigma range. So I would be actually curious if there is anyone who had made some serious testing (not just form 100 games as I did) of cuDNN against other engines and what are the results.

Albert Silver
Posts: 2867
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Tue May 08, 2018 6:21 pm

mirek wrote:
Tue May 08, 2018 12:22 pm
Milos wrote:
Tue May 08, 2018 11:35 am
I played a quick match v0.8 (without TBs) vs v0.7 and they came pretty close, so I don't think there is some bug introduces. Puct was changed from 0.6 to 0.75 but that makes no much difference.
Btw. I also played a match of cuDNN Win version 050518 with batch size 256 on my old GTX 770 vs v0.7, TC=60'+0.6' and cuDNN ended up with 95Elo advantage. Around 5% losses on time, most of them v0.7
There could also possibly be a bug that shows only in in training though.
About cuDNN yes my results are also similar in OCL vs. cuDNN lc0 but when testing against different engines the cuDNN version had only like 20, 30 elo edge, basically still within the 1 sigma range. So I would be actually curious if there is anyone who had made some serious testing (not just form 100 games as I did) of cuDNN against other engines and what are the results.
I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

carldaman
Posts: 1736
Joined: Sat Jun 02, 2012 12:13 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by carldaman » Wed May 09, 2018 4:24 am

Nay Lin Tun wrote:
Tue May 08, 2018 3:05 am
I got this elo graph from someone who come from "Future" https://ibb.co/cxzk67 :mrgreen: :shock:
Not a crazy prediction so far... :|

Ralph Stoesser
Posts: 408
Joined: Sat Mar 06, 2010 8:28 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Ralph Stoesser » Wed May 09, 2018 8:38 am

Seems Leela has fallen in love with Moron :mrgreen:

peter
Posts: 1786
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by peter » Wed May 09, 2018 10:13 am

Hi Albert!
Albert Silver wrote:
Tue May 08, 2018 6:21 pm
I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.
In WAC- tactical postions test-suite with NW 265 from today and the same 24 threads of the 12x3GHz CPU and 10"/move as always, the worst result of all my runs till now.

Having started such with something like NW120 and version 0.7 then, there were about 150 out of 300 poitions solved, best was 159 before making the network bigger, latest till now was 144 and today 140 only.
Peter.

User avatar
Laskos
Posts: 9482
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Wed May 09, 2018 11:14 am

Progress to ID265 with v0.8:

Code: Select all

LCZero v0.8
6s/position 4 CPU threads (equivalent to 1s on GTX 1060)

WAC300 tactical:
ID227: 120/300
ID258: 110/300
ID265: 109/300
performance below 1800 Elo points AB engines, worsening

Openings200 positional:
ID227:  98/200
ID258: 111/200
ID265  114/200
performance above 3300 Elo points AB engines, improving
The eval improvement seems to be going on, comparing LC0 with 1 playout against Komodo 11.3.1 at depth=1:
Image


But strength improvement at fixed time compared to ID227 (first of the larger nets) is lagging behind:
Image

Albert Silver
Posts: 2867
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Wed May 09, 2018 11:18 am

peter wrote:
Wed May 09, 2018 10:13 am
Hi Albert!
Albert Silver wrote:
Tue May 08, 2018 6:21 pm
I had good results, +60 Elo difference, when testing cuDNN (May 5) with puct 3.0 instead of its default 1.7.
In WAC- tactical postions test-suite with NW 265 from today and the same 24 threads of the 12x3GHz CPU and 10"/move as always, the worst result of all my runs till now.

Having started such with something like NW120 and version 0.7 then, there were about 150 out of 300 poitions solved, best was 159 before making the network bigger, latest till now was 144 and today 140 only.
You are aware that the cuDNN versions cannot run on just CPUs? They also have different code and behavior than the normal LCZero. It is not 'just faster'.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

peter
Posts: 1786
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by peter » Wed May 09, 2018 2:55 pm

Albert Silver wrote:
Wed May 09, 2018 11:18 am
You are aware that the cuDNN versions cannot run on just CPUs? They also have different code and behavior than the normal LCZero. It is not 'just faster'.
I have the CPU- version only. My graphic card wouldn't be bad neither, but it's AMD und doesn't support OpenCl.
Peter.

Post Reply