LC0 finally?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Werewolf
Posts: 1795
Joined: Thu Sep 18, 2008 10:24 pm

LC0 finally?

Post by Werewolf »

This looks really promising with the latest networks, is it as good as it looks?
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: LC0 finally?

Post by Laskos »

Werewolf wrote: Wed Jun 20, 2018 10:35 pm This looks really promising with the latest networks, is it as good as it looks?
No. It's not bad generally for 11th net (ID4011), but the rating is misleading. ID4011 is some 500 Elo points weaker than the best nets of the run 1 and run 5 against AB engines, but shows some 400 Elo points better rating in self-games than the best nets of those runs, which is misleading. The rating of the latest run are very inflated, but the real progress against AB engines is still not bad for only 11th net.
Werewolf
Posts: 1795
Joined: Thu Sep 18, 2008 10:24 pm

Re: LC0 finally?

Post by Werewolf »

4018 reads at 3400 elo, so is that about 2900-3000 in real terms?

Not bad....
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: LC0 finally?

Post by Laskos »

Werewolf wrote: Thu Jun 21, 2018 9:58 am 4018 reads at 3400 elo, so is that about 2900-3000 in real terms?

Not bad....
About 2900 in CCRL 40/4' conditions on GTX 1060 GPU. ID4018 is about the level of ID1028 of the previous run (or shown 2500 Elo level of that run), which means a significantly faster progress (18 nets compared to 28 nets, each needing about 40,000 games). The shown rating in self-games now is very inflated, for obscure to me reasons.
crem
Posts: 177
Joined: Wed May 23, 2018 9:29 pm

Re: LC0 finally?

Post by crem »

Laskos wrote: Thu Jun 21, 2018 11:44 am About 2900 in CCRL 40/4' conditions on GTX 1060 GPU. ID4018 is about the level of ID1028 of the previous run (or shown 2500 Elo level of that run), which means a significantly faster progress (18 nets compared to 28 nets, each needing about 40,000 games). The shown rating in self-games now is very inflated, for obscure to me reasons.
I think it's just id4000 vs id4001 match which gave very low precision Elo delta with just 1 lost game (result was +423 -1 =0).
If it was 2 lost games instead of 1, it would be 400 points lower. And if it was 0 lost games instead of 1, it would be +inf points higher.
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: LC0 finally?

Post by yanquis1972 »

is the testserver about ready to switch to a larger net then? think on the first one it was quite quickly; i suppose the thinking is too quickly.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: LC0 finally?

Post by Rebel »

Update -> http://rebel13.nl/zero.html

After the w396 bootstrap lczero (w429) is climbing and climbing again.

Note that the NPS is 33 only.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: LC0 finally?

Post by Laskos »

SzG wrote: Fri Jun 22, 2018 8:57 am Does the graph that I see on the main page indicate that whatever you do this self-learning will saturate at some point so no further progress is to be expected?
By ID4039, in my gauntlet at fast TC against AB engines, it's already similar in strength to best 6x64 nets of the previous runs, scoring 61.0/200. Probably close to saturation, but let's see (if they won't start from scratch again).
crem
Posts: 177
Joined: Wed May 23, 2018 9:29 pm

Re: LC0 finally?

Post by crem »

Some people claim that version 0604-experimental plays stronger than 0619 on the same net. That may theoretically happen both because algorithm differences (I changed some formulas to be able to save some memory and floats may be rounded differently sometimes) or maybe there is nps difference.

I wasn't able to find any differences in my tests though.

Maybe someone has time to do tests and confirm/refute that claim? That would be much appreciated!
JJJ
Posts: 1346
Joined: Sat Apr 19, 2014 1:47 pm

Re: LC0 finally?

Post by JJJ »

Leela seems close to make progress again.