my Lc0 spreadsheet

[Hugo]

I think the Stockfish developement is like a tank that rolls forward.
strong and impressive.

Lc0 strength is unclear to me. Every 24 hrs there are 5 new weightfiles. Its impossible to test them all, maybe some diamonds are undiscovered.
The selfplay Elo seems also to be a hint, but not more.

NN 42000 seems to be a good one, and others have to be found.

C.K.

[jp]

Are we sure Lc0 & A0 use the same definition of "node"?

No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?

We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.

[Laskos]

Are we sure Lc0 & A0 use the same definition of "node"?

No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?

We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.

Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.

[Hugo]

I am having problems to gain any performance plus out of using two RTX 2060 cards.
Tried 3 different backends but all the same.
nodes are ~ 45.000+ nps after a minute or two in startpossition.
But in 5m +3s games I cannot see any - ANY! - better results yet.
Tried multiplex, roundrobin and demux. 15 - 20 games.
Let run demux over night.

C.K.

[jp]

We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do.

Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.

Yes, it's not so practically important, but it'd be nice to know with some confidence that Lc is stronger than A0, since Lc is all we have.

[Hugo]

Hi all

first testrun (100games, 5m+3s, ponder ON) with 2 GPU is done. At the end it was a plus of 28 Elo (Elostat) or 29,3 Elo (Ordo).
backend was multiplexing.
I start a new testrun with backend=roundrobin.
Then (two days) I will continue testing on single RTX 2060.

Code: Select all

  # PLAYER                          : RATING    POINTS  PLAYED    (%)
   1 Lc0 v0.21.1-42000-x2GPU-mp      :   29.3      54.0     100   54.0%
   2 Stockfish 100419 64 BMI2-x12    :    0.0     526.0    1000   52.6%
   3 Lc0 v0.21.1-42000               :    0.0     100.0     200   50.0%
   4 Lc0 v0.21.1-41997               :  -14.6      48.0     100   48.0%
   5 Lc0 v0.21.1-42029               :  -25.6      46.5     100   46.5%
   6 Lc0 v0.21.1-41812               :  -25.6      46.5     100   46.5%
   7 Lc0 v0.21.1-41965               :  -29.3      46.0     100   46.0%
   8 Lc0 v0.21.1-42043               :  -29.3      46.0     100   46.0%
   9 Lc0 v0.21.1-41889               :  -40.3      44.5     100   44.5%
  10 Lc0 v0.21.1-41906               :  -55.1      42.5     100   42.5%

https://docs.google.com/spreadsheets/d/ ... sp=sharing

kind regards, C.K.

[Hugo]

Hi all

two new tests added. NN 42070 had a good result and is on the rematch.
Top scorer still 42000.

regards, C.K.

Code: Select all

   # PLAYER                          : RATING    POINTS  PLAYED    (%)
   1 Lc0 v0.21.1-42000-x2GPU-mp      :   29.3      54.0     100   54.0%
   2 Stockfish 100419 64 BMI2-x12    :    0.0     629.0    1200   52.4%
   3 Lc0 v0.21.1-42000               :    0.0     100.0     200   50.0%
   4 Lc0 v0.21.1-42070               :   -3.7      49.5     100   49.5%
   5 Lc0 v0.21.1-41997               :  -14.6      48.0     100   48.0%
   6 Lc0 v0.21.1-42107               :  -18.3      47.5     100   47.5%
   7 Lc0 v0.21.1-42029               :  -25.6      46.5     100   46.5%
   8 Lc0 v0.21.1-41812               :  -25.6      46.5     100   46.5%
   9 Lc0 v0.21.1-42043               :  -29.3      46.0     100   46.0%
  10 Lc0 v0.21.1-41965               :  -29.3      46.0     100   46.0%
  11 Lc0 v0.21.1-41889               :  -40.3      44.5     100   44.5%
  12 Lc0 v0.21.1-41906               :  -55.1      42.5     100   42.5%

https://docs.google.com/spreadsheets/d/ ... sp=sharing

[Hugo]

Hi All
actually testing 180 games

Stockfish 100419 64 BMI2-x12 vs Lc0 v0.21.1-T40.T8.610

5m+3s , ponder ON.

after 116 games Lc0 is 14 games in front !

+26 =78 -12 from sight of Lc0.

How will it end afte 180 games?

regards, C.K.

[whereagles]

kai.. you now look like robert de niro in taxi driver

[Hugo]

after two weeks of break, I started a new test run with nn 42441 which has the highest selplay Elo atm.

last sunday I could win the Engine Masters Tournament on Infinitychess in front of 36 players with a tripple GPU nn 42232 / 91.000 nps.

regards, C.K.

https://docs.google.com/spreadsheets/d/ ... sp=sharing