Page 2 of 3
Re: my Lc0 spreadsheet
Posted: Tue Apr 23, 2019 10:40 pm
by Hugo
I think the Stockfish developement is like a tank that rolls forward.
strong and impressive.
Lc0 strength is unclear to me. Every 24 hrs there are 5 new weightfiles. Its impossible to test them all, maybe some diamonds are undiscovered.
The selfplay Elo seems also to be a hint, but not more.
NN 42000 seems to be a good one, and others have to be found.
C.K.
Re: my Lc0 spreadsheet
Posted: Tue Apr 23, 2019 11:09 pm
by jp
Laskos wrote: ↑Tue Apr 23, 2019 8:04 pm
jp wrote: ↑Tue Apr 23, 2019 4:41 pm
Are we sure Lc0 & A0 use the same definition of "node"?
No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.
Re: my Lc0 spreadsheet
Posted: Tue Apr 23, 2019 11:21 pm
by Laskos
jp wrote: ↑Tue Apr 23, 2019 11:09 pm
Laskos wrote: ↑Tue Apr 23, 2019 8:04 pm
jp wrote: ↑Tue Apr 23, 2019 4:41 pm
Are we sure Lc0 & A0 use the same definition of "node"?
No, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.
Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.
Re: my Lc0 spreadsheet
Posted: Wed Apr 24, 2019 9:21 pm
by Hugo
I am having problems to gain any performance plus out of using two RTX 2060 cards.
Tried 3 different backends but all the same.
nodes are ~ 45.000+ nps after a minute or two in startpossition.
But in 5m +3s games I cannot see any - ANY! - better results yet.
Tried multiplex, roundrobin and demux. 15 - 20 games.
Let run demux over night.
C.K.
Re: my Lc0 spreadsheet
Posted: Thu Apr 25, 2019 4:49 pm
by jp
Laskos wrote: ↑Tue Apr 23, 2019 11:21 pm
jp wrote: ↑Tue Apr 23, 2019 11:09 pm
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do.
Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.
Yes, it's not so practically important, but it'd be nice to know with some confidence that Lc is stronger than A0, since Lc is all we have.
Re: my Lc0 spreadsheet
Posted: Fri Apr 26, 2019 3:30 pm
by Hugo
Hi all
first testrun (100games, 5m+3s, ponder ON) with 2 GPU is done. At the end it was a plus of 28 Elo (Elostat) or 29,3 Elo (Ordo).
backend was multiplexing.
I start a new testrun with backend=roundrobin.
Then (two days) I will continue testing on single RTX 2060.
Code: Select all
# PLAYER : RATING POINTS PLAYED (%)
1 Lc0 v0.21.1-42000-x2GPU-mp : 29.3 54.0 100 54.0%
2 Stockfish 100419 64 BMI2-x12 : 0.0 526.0 1000 52.6%
3 Lc0 v0.21.1-42000 : 0.0 100.0 200 50.0%
4 Lc0 v0.21.1-41997 : -14.6 48.0 100 48.0%
5 Lc0 v0.21.1-42029 : -25.6 46.5 100 46.5%
6 Lc0 v0.21.1-41812 : -25.6 46.5 100 46.5%
7 Lc0 v0.21.1-41965 : -29.3 46.0 100 46.0%
8 Lc0 v0.21.1-42043 : -29.3 46.0 100 46.0%
9 Lc0 v0.21.1-41889 : -40.3 44.5 100 44.5%
10 Lc0 v0.21.1-41906 : -55.1 42.5 100 42.5%
kind regards, C.K.
Re: my Lc0 spreadsheet
Posted: Sun Apr 28, 2019 6:05 pm
by Hugo
Hi all
two new tests added. NN 42070 had a good result and is on the rematch.
Top scorer still 42000.
regards, C.K.
Code: Select all
# PLAYER : RATING POINTS PLAYED (%)
1 Lc0 v0.21.1-42000-x2GPU-mp : 29.3 54.0 100 54.0%
2 Stockfish 100419 64 BMI2-x12 : 0.0 629.0 1200 52.4%
3 Lc0 v0.21.1-42000 : 0.0 100.0 200 50.0%
4 Lc0 v0.21.1-42070 : -3.7 49.5 100 49.5%
5 Lc0 v0.21.1-41997 : -14.6 48.0 100 48.0%
6 Lc0 v0.21.1-42107 : -18.3 47.5 100 47.5%
7 Lc0 v0.21.1-42029 : -25.6 46.5 100 46.5%
8 Lc0 v0.21.1-41812 : -25.6 46.5 100 46.5%
9 Lc0 v0.21.1-42043 : -29.3 46.0 100 46.0%
10 Lc0 v0.21.1-41965 : -29.3 46.0 100 46.0%
11 Lc0 v0.21.1-41889 : -40.3 44.5 100 44.5%
12 Lc0 v0.21.1-41906 : -55.1 42.5 100 42.5%
Re: my Lc0 spreadsheet
Posted: Fri May 10, 2019 4:44 pm
by Hugo
Hi All
actually testing 180 games
Stockfish 100419 64 BMI2-x12 vs Lc0 v0.21.1-T40.T8.610
5m+3s , ponder ON.
after 116 games Lc0 is 14 games in front !
+26 =78 -12 from sight of Lc0.
How will it end afte 180 games?
regards, C.K.
Re: my Lc0 spreadsheet
Posted: Sat May 11, 2019 11:39 pm
by whereagles
kai.. you now look like robert de niro in taxi driver
Re: my Lc0 spreadsheet
Posted: Sun Jun 02, 2019 11:45 am
by Hugo
after two weeks of break, I started a new test run with nn 42441 which has the highest selplay Elo atm.
last sunday I could win the Engine Masters Tournament on Infinitychess in front of 36 players with a tripple GPU nn 42232 / 91.000 nps.
regards, C.K.
https://docs.google.com/spreadsheets/d/ ... sp=sharing