my Lc0 spreadsheet

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Tue Apr 23, 2019 8:40 pm

I think the Stockfish developement is like a tank that rolls forward.
strong and impressive.

Lc0 strength is unclear to me. Every 24 hrs there are 5 new weightfiles. Its impossible to test them all, maybe some diamonds are undiscovered.
The selfplay Elo seems also to be a hint, but not more.

NN 42000 seems to be a good one, and others have to be found.

C.K.

jp
Posts: 755
Joined: Mon Apr 23, 2018 5:54 am

Re: my Lc0 spreadsheet

Post by jp » Tue Apr 23, 2019 9:09 pm

Laskos wrote:
Tue Apr 23, 2019 6:04 pm
jp wrote:
Tue Apr 23, 2019 2:41 pm
Are we sure Lc0 & A0 use the same definition of "node"?
No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.

User avatar
Laskos
Posts: 9414
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: my Lc0 spreadsheet

Post by Laskos » Tue Apr 23, 2019 9:21 pm

jp wrote:
Tue Apr 23, 2019 9:09 pm
Laskos wrote:
Tue Apr 23, 2019 6:04 pm
jp wrote:
Tue Apr 23, 2019 2:41 pm
Are we sure Lc0 & A0 use the same definition of "node"?
No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.
Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Wed Apr 24, 2019 7:21 pm

I am having problems to gain any performance plus out of using two RTX 2060 cards.
Tried 3 different backends but all the same.
nodes are ~ 45.000+ nps after a minute or two in startpossition.
But in 5m +3s games I cannot see any - ANY! - better results yet.
Tried multiplex, roundrobin and demux. 15 - 20 games.
Let run demux over night.

C.K.

jp
Posts: 755
Joined: Mon Apr 23, 2018 5:54 am

Re: my Lc0 spreadsheet

Post by jp » Thu Apr 25, 2019 2:49 pm

Laskos wrote:
Tue Apr 23, 2019 9:21 pm
jp wrote:
Tue Apr 23, 2019 9:09 pm
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do.
Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.
Yes, it's not so practically important, but it'd be nice to know with some confidence that Lc is stronger than A0, since Lc is all we have.

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Fri Apr 26, 2019 1:30 pm

Hi all

first testrun (100games, 5m+3s, ponder ON) with 2 GPU is done. At the end it was a plus of 28 Elo (Elostat) or 29,3 Elo (Ordo).
backend was multiplexing.
I start a new testrun with backend=roundrobin.
Then (two days) I will continue testing on single RTX 2060.

Code: Select all

  # PLAYER                          : RATING    POINTS  PLAYED    (%)
   1 Lc0 v0.21.1-42000-x2GPU-mp      :   29.3      54.0     100   54.0%
   2 Stockfish 100419 64 BMI2-x12    :    0.0     526.0    1000   52.6%
   3 Lc0 v0.21.1-42000               :    0.0     100.0     200   50.0%
   4 Lc0 v0.21.1-41997               :  -14.6      48.0     100   48.0%
   5 Lc0 v0.21.1-42029               :  -25.6      46.5     100   46.5%
   6 Lc0 v0.21.1-41812               :  -25.6      46.5     100   46.5%
   7 Lc0 v0.21.1-41965               :  -29.3      46.0     100   46.0%
   8 Lc0 v0.21.1-42043               :  -29.3      46.0     100   46.0%
   9 Lc0 v0.21.1-41889               :  -40.3      44.5     100   44.5%
  10 Lc0 v0.21.1-41906               :  -55.1      42.5     100   42.5%
https://docs.google.com/spreadsheets/d/ ... sp=sharing

kind regards, C.K.

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Sun Apr 28, 2019 4:05 pm

Hi all

two new tests added. NN 42070 had a good result and is on the rematch.
Top scorer still 42000.

regards, C.K.

Code: Select all

   # PLAYER                          : RATING    POINTS  PLAYED    (%)
   1 Lc0 v0.21.1-42000-x2GPU-mp      :   29.3      54.0     100   54.0%
   2 Stockfish 100419 64 BMI2-x12    :    0.0     629.0    1200   52.4%
   3 Lc0 v0.21.1-42000               :    0.0     100.0     200   50.0%
   4 Lc0 v0.21.1-42070               :   -3.7      49.5     100   49.5%
   5 Lc0 v0.21.1-41997               :  -14.6      48.0     100   48.0%
   6 Lc0 v0.21.1-42107               :  -18.3      47.5     100   47.5%
   7 Lc0 v0.21.1-42029               :  -25.6      46.5     100   46.5%
   8 Lc0 v0.21.1-41812               :  -25.6      46.5     100   46.5%
   9 Lc0 v0.21.1-42043               :  -29.3      46.0     100   46.0%
  10 Lc0 v0.21.1-41965               :  -29.3      46.0     100   46.0%
  11 Lc0 v0.21.1-41889               :  -40.3      44.5     100   44.5%
  12 Lc0 v0.21.1-41906               :  -55.1      42.5     100   42.5%



https://docs.google.com/spreadsheets/d/ ... sp=sharing

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Fri May 10, 2019 2:44 pm

Hi All
actually testing 180 games

Stockfish 100419 64 BMI2-x12 vs Lc0 v0.21.1-T40.T8.610

5m+3s , ponder ON.

after 116 games Lc0 is 14 games in front !

+26 =78 -12 from sight of Lc0.

How will it end afte 180 games?

regards, C.K.

whereagles
Posts: 560
Joined: Thu Nov 13, 2014 11:03 am

Re: my Lc0 spreadsheet

Post by whereagles » Sat May 11, 2019 9:39 pm

kai.. you now look like robert de niro in taxi driver :D

Hugo
Posts: 775
Joined: Tue Dec 01, 2009 10:10 am

Re: my Lc0 spreadsheet

Post by Hugo » Sun Jun 02, 2019 9:45 am

after two weeks of break, I started a new test run with nn 42441 which has the highest selplay Elo atm.

last sunday I could win the Engine Masters Tournament on Infinitychess in front of 36 players with a tripple GPU nn 42232 / 91.000 nps.

regards, C.K.

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Post Reply