Houston: We have lift off ...

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
whereagles
Posts: 560
Joined: Thu Nov 13, 2014 11:03 am

Re: Houston: We have lift off ...

Post by whereagles » Mon Nov 19, 2018 3:45 pm

many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)

chrisw
Posts: 1723
Joined: Tue Apr 03, 2012 2:28 pm

Re: Houston: We have lift off ...

Post by chrisw » Mon Nov 19, 2018 4:05 pm

whereagles wrote:
Mon Nov 19, 2018 3:45 pm
many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)
what's the network size for 11248?

User avatar
Leto
Posts: 2028
Joined: Thu May 04, 2006 1:40 am
Location: Dune

Re: Houston: We have lift off ...

Post by Leto » Mon Nov 19, 2018 6:22 pm

Laskos wrote:
Mon Nov 19, 2018 5:39 am
Leto wrote:
Mon Nov 19, 2018 12:18 am
Laskos wrote:
Fri Nov 16, 2018 2:56 pm
I don't know why you are so enthusiastic. Runs 20xxx and 30xxx are pretty pathetic, especially considering how much resources they have eaten up. Some folks there have overdone something. Just a quick check with the latest engine (rc4) and one of the latest nets:

TC: 60'' + 1''

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
     SF8                           120      68      60   66.7%   43.3%
   
   1 lc0_v19_11261                   0     111      20   50.0%   50.0%
   2 lc0_v19_31214                -147     128      20   30.0%   40.0%
   3 lc0_v19_9155                 -241     127      20   20.0%   40.0%
Finished match
So, run 30xxx is still ~150 Elo points below run 10xxx, and barely ~100 Elo points above 6x64 net 9155 (run 9xxx). Taking into account that the games with 6x64 net were 10-12 times faster and taking into account the hardware resources allocated, the whole run 9xxx could have been completed in less than a day. Lame runs, these newest ones. But I still hope that they will improve some 200 real Elo points over current level, although this is not granted at all.
I don't think Test30 is this close to Test10 in strength, I still think it's several hundred elo weaker. What's 60" + 1", is that game in 1 minute with an extra second per move?
Yes, 1m + 1s. It is close, Test30 is about 100 Elo points weaker than Test10.
I highly doubt that. I have it at just slightly stronger than Stockfish 5 1CPU at 1 minute blitz and my tests run 200 games each. The best I've gotten from a test30 net is a 59% score which would put it about 60 elo stronger than SF5 1CPU on my machine (Ryzen 5 2600 with Nvidia 1080). If I had a 2080ti maybe it would perform about 100 elo higher on my machine than SF5 but that would still not put it anywhere near the best Test10 net 11250 which is about as strong as Stockfish 9.

See this chart, it has 11250 between 200 and 300 elo stronger than the current test30 networks: https://docs.google.com/spreadsheets/d/ ... =952456918

jp
Posts: 643
Joined: Mon Apr 23, 2018 5:54 am

Re: Houston: We have lift off ...

Post by jp » Mon Nov 19, 2018 8:19 pm

chrisw wrote:
Mon Nov 19, 2018 4:05 pm
whereagles wrote:
Mon Nov 19, 2018 3:45 pm
many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)
what's the network size for 11248?
Same size. Only 9xxx are smaller.

chrisw
Posts: 1723
Joined: Tue Apr 03, 2012 2:28 pm

Re: Houston: We have lift off ...

Post by chrisw » Mon Nov 19, 2018 8:40 pm

jp wrote:
Mon Nov 19, 2018 8:19 pm
chrisw wrote:
Mon Nov 19, 2018 4:05 pm
whereagles wrote:
Mon Nov 19, 2018 3:45 pm
many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)
what's the network size for 11248?
Same size. Only 9xxx are smaller.
sorry to be a pain, but what size is same size?

carldaman
Posts: 1675
Joined: Sat Jun 02, 2012 12:13 am

Re: Houston: We have lift off ...

Post by carldaman » Mon Nov 19, 2018 11:28 pm

Where can network 11248 be downloaded?
Thanks! :)

User avatar
CMCanavessi
Posts: 832
Joined: Thu Dec 28, 2017 3:06 pm
Location: Argentina

Re: Houston: We have lift off ...

Post by CMCanavessi » Tue Nov 20, 2018 12:13 am

chrisw wrote:
Mon Nov 19, 2018 8:40 pm
jp wrote:
Mon Nov 19, 2018 8:19 pm
chrisw wrote:
Mon Nov 19, 2018 4:05 pm
whereagles wrote:
Mon Nov 19, 2018 3:45 pm
many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)
what's the network size for 11248?
Same size. Only 9xxx are smaller.
sorry to be a pain, but what size is same size?
20 blocks, 256 filters
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls

chrisw
Posts: 1723
Joined: Tue Apr 03, 2012 2:28 pm

Re: Houston: We have lift off ...

Post by chrisw » Tue Nov 20, 2018 1:20 am

CMCanavessi wrote:
Tue Nov 20, 2018 12:13 am
chrisw wrote:
Mon Nov 19, 2018 8:40 pm
jp wrote:
Mon Nov 19, 2018 8:19 pm
chrisw wrote:
Mon Nov 19, 2018 4:05 pm
whereagles wrote:
Mon Nov 19, 2018 3:45 pm
many are testing 30xxx vs 11248.. seems 11248 is finally within reach (30 elo away)
what's the network size for 11248?
Same size. Only 9xxx are smaller.
sorry to be a pain, but what size is same size?
20 blocks, 256 filters
thanks. now I know this “good” version was with a large net.

ok, the point I’ld like to make is this:
LC0 puts out a new net often, in a continuous stream, is quite a while since I downloaded one, but IIRC there were new nets every day. they get kind of tested at LC0 HQ in self play, but also tested kind of randomly by end users. With occasional post reports about how net XYZ is amazing or is crud depending, probably on the phase of the moon. Now if there are random tests on a stream of nets, there is also noise being testing. Net A0054 might have some fortuitous noise that causes it to play better, but when Net A0054 gets a bit of training and becomes Net A0055 the fortuitous noise changes and Net0055 doesn’t seem to perform so well, maybe, and so it goes on. What am i saying? Ok, there’s a difference between fortuitous noise that gives a higher ego signal, but is then lost, because it was only noise, and stable learnt weights which don’t lose their “knowledge” because that stable knowledge is genuinely there, and not lost with more “learning” twiddling. So, in any learning run, there will will random elo noise, and its no good pointing at, say 11345, and holding its elo as a target, if 11346 and 11347 and so on don’t hold the headline elo that 11345 achieved. If 11345 successors can’t hold the headline elo, then that elo was just fortuitous noise, not anything stably learnt.

You can’t just pick one net out of a series and claim the series is “best”, you need to show stability in the series in general. I don’t think that’s been shown, for any net series actually. Occasional headline glitches don’t mean anything, if they can’t be held, they’re not there.

MikeB
Posts: 3189
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Houston: We have lift off ...

Post by MikeB » Tue Nov 20, 2018 4:14 am

carldaman wrote:
Mon Nov 19, 2018 11:28 pm
Where can network 11248 be downloaded?
Thanks! :)
main network site
https://lczero.org/networks/

do search for 11248 or just click the link below'
https://lczero.org/get_network?sha=994d ... 74419caecd

I would grab 11260 as well..
https://lczero.org/get_network?sha=7c06 ... f910af8841

jp
Posts: 643
Joined: Mon Apr 23, 2018 5:54 am

Re: Houston: We have lift off ...

Post by jp » Tue Nov 20, 2018 5:29 am

chrisw wrote:
Mon Nov 19, 2018 8:40 pm
jp wrote:
Mon Nov 19, 2018 8:19 pm
chrisw wrote:
Mon Nov 19, 2018 4:05 pm
what's the network size for 11248?
Same size. Only 9xxx are smaller.
sorry to be a pain, but what size is same size?
1xxxx, 2xxxx, 3xxxx are all 20x256.
9xxx are 6x64.

(Just seeing more recent posts now.)

Post Reply