LCZero: Progress and Scaling. Relation to CCRL Elo

[CMCanavessi]

Thanks, very interesting, I hope you make the people involved aware of your work.

Oh they are aware of course, as I always broadcast the gauntlets live

103 is now about to start running the test... click my signature if you want to follow the games.

[Jhoravi]

A very curious match is between ID83 and ID103 because they're so close in ELO rating despite ID103 is more experienced with around 1 million more playoffs.

[Laskos]

Thanks, very interesting, I hope you make the people involved aware of your work.

Oh they are aware of course, as I always broadcast the gauntlets live

103 is now about to start running the test... click my signature if you want to follow the games.

Thanks, it seems it's heading for a good result, close to 130/200 compared to ID98 113/200. Maybe Elo of 1970-80, compared to Elo of 1906.

With my positional opening test suite of 200 positions, I have a large progress too, at this pace LC0 in less than a month from now will be among the very top engines there:

Code: Select all

[Search parameters: MaxDepth=99   MaxTime=20.0   DepthDelta=2   MinDepth=7   MinTime=0.1] 

Engine                         : Correct  TotalPos  Corr%  AveT(s)  MaxT(s)  TestFile 
      
Komodo 10.2 64-bit             :     145       200   72.5      2.0     20.0  openings200beta07.epd 
Houdini 5.01 Pro x64           :     144       200   72.0      2.4     20.0  openings200beta07.epd    
Stockfish 8 64 BMI2            :     141       200   70.5      2.0     20.0  openings200beta07.epd 
Houdini 5.01 Pro x64 Tactical  :     139       200   69.5      2.3     20.0  openings200beta07.epd      
Deep Shredder 13 x64           :     128       200   64.0      2.7     20.0  openings200beta07.epd     
Andscacs 0.88n                 :     123       200   61.5      2.4     20.0  openings200beta07.epd 
Houdini 4 Pro    (3335 CCRL)   :     120       200   60.0      1.6     20.0  openings200beta07.epd 
Nirvanachess 2.3 (3216 CCRL)   :     119       200   59.5      1.8     20.0  openings200beta07.epd 

LCZero  *************  ID103   :     115       200   57.5      1.7     20.0  openings200beta07.epd

Fire 5 x64       (3341 CCRL)   :     110       200   55.0      3.0     20.0  openings200beta07.epd    
Texel 1.06       (3162 CCRL)   :     110       200   55.0      1.6     20.0  openings200beta07.epd    

LCZero  *************  ID83    :     109       200   54.5      1.1     20.0  openings200beta07.epd

Fritz 15         (3227 CCRL)   :     102       200   51.0      1.9     20.0  openings200beta07.epd  

LCZero  *************  ID69    :      98       200   49.0      2.7     20.0  openings200beta07.epd 
  
Fruit 2.1        (2685 CCRL)   :      91       200   45.5      1.5     20.0  openings200beta07.epd  

LCZero  *************  ID59    :      90       200   45.0      1.7     20.0  openings200beta07.epd

Sjaak II 1.3.1   (2194 CCRL)   :      75       200   37.5      4.0     20.0  openings200beta07.epd    
BikJump v2.01    (2098 CCRL)   :      74       200   37.0      1.6     20.0  openings200beta07.epd

[stavros]

I've made a graph with my results and gauntlets which clearly shows that there's still no stall in the strenght's grow:

Elo progression:



Gauntlet score progression:

very nice graph but what is the difference with https://docs.google.com/spreadsheets/d/ ... edit#gid=0

big elo difference which is the correct?

[JJJ]

Yeah, more progress today !

[Werewolf]

Thanks, very interesting, I hope you make the people involved aware of your work.

Oh they are aware of course, as I always broadcast the gauntlets live

103 is now about to start running the test... click my signature if you want to follow the games.

Stop improving so fast, it's making it hard to test!!

[CMCanavessi]

Updated graphics with the recent gauntlet:

[CMCanavessi]

very nice graph but what is the difference with https://docs.google.com/spreadsheets/d/ ... edit#gid=0

big elo difference which is the correct?

It's a completely different approach. He only tests vs. stockfish, I test vs a lot of different engines (every gauntlet it plays vs 25 other engines).

Also, he uses Stockfish 8 = 3355 elo as anchor.
I use Teki Random mover = 0 and build the ratings from there.

You can see my full rating list here: https://pastebin.com/9Mg8nBvy

[stavros]

very nice graph but what is the difference with https://docs.google.com/spreadsheets/d/ ... edit#gid=0

big elo difference which is the correct?

It's a completely different approach. He only tests vs. stockfish, I test vs a lot of different engines (every gauntlet it plays vs 25 other engines).

Also, he uses Stockfish 8 = 3355 elo as anchor.
I use Teki Random mover = 0 and build the ratings from there.

You can see my full rating list here: https://pastebin.com/9Mg8nBvy

i got it ,how about your kn/s (lczero)? also whats the elo gain for every doubling speed of lczero? tnx

[CMCanavessi]

I get about 1kn/s, and tc is 1 min + 1 sec so it's very similar to the match games leela plays between networks.

The scaling, i don't know, haven't made any tests.