Super bullet chess: LC0 v0.21.2-rc1 network 3 vs newest Stockfish dev 170519

[Hai]

Super bullet chess: LC0 v0.21.2-rc1 network 3 vs newest Stockfish dev 170519

The 20 possible opening moves
40 games
Ponder on
LC0 with 2x RTX 2080 Ti
Stockfish with 1 core
Time control: 1 min per game + 1 second per move.

Result:
LC0 29 points
Stockfish 11 points
+20 =18 -2
Winning percentage = 72.50%

= LC0 is 168 elo stronger than Stockfish.
http://www.mediafire.com/file/a58z864se ... 9.pgn/file
That also means you will need a 16 core amd threadripper 2950X to draw or win against this 40 x 256 network.
Note that you will need after some more networks a 32 core amd threadripper 2990WX to draw or win against this 40 x 256 network.


What is network 3?
It's a 40 x 256 network.
http://157.230.189.191:8080/networks/
http://157.230.189.191:8080/
https://groups.google.com/forum/#!topic ... 5m-2JCdo-A

I hope it will be possible to support also the 40 x 512 network and not only the 40 x 256 network.
40 x 512 could be ready today or tomorrow or in the next time:)

Also note that you can have a depth between 1 and 6 and still easily kill Stockfish .

[lkaufman]

So far it seems that network 3 is not at all competitive with the best regular Lc0 network (I'm using 42372, 2080 gpu, 2' + 1"). Am I missing something? Is there some reason to think network 3 is actually useful now? Running it on a 2080 vs SF on 1 core is silly, that doesn't tell us anything.

[ernest]

Do you even realize that your Leela ratio is perhaps as high as 10 ?...

So what's the use ? Very disappointing, Hai !

[Hai]

So far it seems that network 3 is not at all competitive with the best regular Lc0 network (I'm using 42372, 2080 gpu, 2' + 1"). Am I missing something? Is there some reason to think network 3 is actually useful now? Running it on a 2080 vs SF on 1 core is silly, that doesn't tell us anything.

Of course network 3 is not competitive with the best regular LC0 network.
You are missing:
1. From 3 to 42372 you have 42369 other networks, due to this fact 42372 is at the moment obviously stronger than 3.
2. Network 3 is 40 x 256 and 42372 is 20 x 256
3. You can test new network 3 (40 x 256) vs old network 3 or vs 40003 (20 x 256)
4. Of course it is actually somehow useful. It's weaker but Komodo, Houdini and Stockfish are weaker too compared to 42372.

[Hai]

Do you even realize that your Leela ratio is perhaps as high as 10 ?...

So what's the use ? Very disappointing, Hai !

1. There is no reason not to test a new 40 x 256 network first vs 1 core Stockfish, then 2 cores, then 4 cores, then 8 cores...or due I miss some law that everybody must run matches vs at least 32 core Stockfish but better vs 64 core Stockfish or even better vs 128 core Stockfish?
2. I'm not into the stupid ratio stuff.
3. But think yourself of what is the ratio when:
42372 have a ratio of maybe 10 vs 1 core Stockfish on my hardware, but network 3 is compared to 42372(~1 year old) like 1 day vs 365 days.
Calculating this into your Leela ratio, I see now that Stockfish was way to much in the advantage.

[lkaufman]

So far it seems that network 3 is not at all competitive with the best regular Lc0 network (I'm using 42372, 2080 gpu, 2' + 1"). Am I missing something? Is there some reason to think network 3 is actually useful now? Running it on a 2080 vs SF on 1 core is silly, that doesn't tell us anything.

Of course network 3 is not competitive with the best regular LC0 network.
You are missing:
1. From 3 to 42372 you have 42369 other networks, due to this fact 42372 is at the moment obviously stronger than 3.
2. Network 3 is 40 x 256 and 42372 is 20 x 256
3. You can test new network 3 (40 x 256) vs old network 3 or vs 40003 (20 x 256)
4. Of course it is actually somehow useful. It's weaker but Komodo, Houdini and Stockfish are weaker too compared to 42372.

OK, so your point is that it is very good for a newborn engine. But I note that network one was shown as having a 3100 rating. Did it start from zero or from some already strong network?

[dkappe]

1. From 3 to 42372 you have 42369 other networks, due to this fact 42372 is at the moment obviously stronger than 3.

You do understand that these network id’s are unique identifiers in the lczero database? So, given that t40 started at 40000, I’ll leave it to the reader to correct the arithmetic.

[lkaufman]

So far it seems that network 20 is now stronger than the best of the 10k series on my 2080. That doesn't make it competitve with the best of the 40k series yet, but it does make it look pretty promising. I would expect it to scale better with more time, so it is even possible that network 20 is already better than the 40k best at some very long time control, but probably not yet.