I think the Stockfish developement is like a tank that rolls forward.
strong and impressive.
Lc0 strength is unclear to me. Every 24 hrs there are 5 new weightfiles. Its impossible to test them all, maybe some diamonds are undiscovered.
The selfplay Elo seems also to be a hint, but not more.
NN 42000 seems to be a good one, and others have to be found.
C.K.
my Lc0 spreadsheet
Moderators: hgm, Rebel, chrisw
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: my Lc0 spreadsheet
We would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.Laskos wrote: ↑Tue Apr 23, 2019 8:04 pmNo, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: my Lc0 spreadsheet
Well, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.jp wrote: ↑Tue Apr 23, 2019 11:09 pmWe would be able to check if the A0 guys stated what their hardware was for SF in their paper, but I don't think they do. They give very little info about anything.Laskos wrote: ↑Tue Apr 23, 2019 8:04 pmNo, but it's quite possible. Also, I was thinking of the upcoming TCEC superfinal, where the effective Leela Ratio (as defined in the paper) is close to 1.0. According to my results, Lc0 t40 should win. According to the results of Hugo, it is very doubtful. I would make a bet on Lc0, are bettings open somewhere on TCEC sites?
The comments in the other thread suggest the Lc0 "nps" definition is flattering to Lc0, because they are counting only terminal nodes.
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
Re: my Lc0 spreadsheet
I am having problems to gain any performance plus out of using two RTX 2060 cards.
Tried 3 different backends but all the same.
nodes are ~ 45.000+ nps after a minute or two in startpossition.
But in 5m +3s games I cannot see any - ANY! - better results yet.
Tried multiplex, roundrobin and demux. 15 - 20 games.
Let run demux over night.
C.K.
Tried 3 different backends but all the same.
nodes are ~ 45.000+ nps after a minute or two in startpossition.
But in 5m +3s games I cannot see any - ANY! - better results yet.
Tried multiplex, roundrobin and demux. 15 - 20 games.
Let run demux over night.
C.K.
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
Re: my Lc0 spreadsheet
Yes, it's not so practically important, but it'd be nice to know with some confidence that Lc is stronger than A0, since Lc is all we have.Laskos wrote: ↑Tue Apr 23, 2019 11:21 pmWell, after all maybe it's not that important, but more important is that my GPU seems stronger NPS-wise than one of their TPU from the paper (IIRC they used 4 first generation TPUs in games against SF), even if maybe nodes are different. This seemed a bit science-fiction to me only a year ago, when I was still using OpenCL with my CPU.
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
Re: my Lc0 spreadsheet
Hi all
first testrun (100games, 5m+3s, ponder ON) with 2 GPU is done. At the end it was a plus of 28 Elo (Elostat) or 29,3 Elo (Ordo).
backend was multiplexing.
I start a new testrun with backend=roundrobin.
Then (two days) I will continue testing on single RTX 2060.
https://docs.google.com/spreadsheets/d/ ... sp=sharing
kind regards, C.K.
first testrun (100games, 5m+3s, ponder ON) with 2 GPU is done. At the end it was a plus of 28 Elo (Elostat) or 29,3 Elo (Ordo).
backend was multiplexing.
I start a new testrun with backend=roundrobin.
Then (two days) I will continue testing on single RTX 2060.
Code: Select all
# PLAYER : RATING POINTS PLAYED (%)
1 Lc0 v0.21.1-42000-x2GPU-mp : 29.3 54.0 100 54.0%
2 Stockfish 100419 64 BMI2-x12 : 0.0 526.0 1000 52.6%
3 Lc0 v0.21.1-42000 : 0.0 100.0 200 50.0%
4 Lc0 v0.21.1-41997 : -14.6 48.0 100 48.0%
5 Lc0 v0.21.1-42029 : -25.6 46.5 100 46.5%
6 Lc0 v0.21.1-41812 : -25.6 46.5 100 46.5%
7 Lc0 v0.21.1-41965 : -29.3 46.0 100 46.0%
8 Lc0 v0.21.1-42043 : -29.3 46.0 100 46.0%
9 Lc0 v0.21.1-41889 : -40.3 44.5 100 44.5%
10 Lc0 v0.21.1-41906 : -55.1 42.5 100 42.5%
kind regards, C.K.
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
Re: my Lc0 spreadsheet
Hi all
two new tests added. NN 42070 had a good result and is on the rematch.
Top scorer still 42000.
regards, C.K.
https://docs.google.com/spreadsheets/d/ ... sp=sharing
two new tests added. NN 42070 had a good result and is on the rematch.
Top scorer still 42000.
regards, C.K.
Code: Select all
# PLAYER : RATING POINTS PLAYED (%)
1 Lc0 v0.21.1-42000-x2GPU-mp : 29.3 54.0 100 54.0%
2 Stockfish 100419 64 BMI2-x12 : 0.0 629.0 1200 52.4%
3 Lc0 v0.21.1-42000 : 0.0 100.0 200 50.0%
4 Lc0 v0.21.1-42070 : -3.7 49.5 100 49.5%
5 Lc0 v0.21.1-41997 : -14.6 48.0 100 48.0%
6 Lc0 v0.21.1-42107 : -18.3 47.5 100 47.5%
7 Lc0 v0.21.1-42029 : -25.6 46.5 100 46.5%
8 Lc0 v0.21.1-41812 : -25.6 46.5 100 46.5%
9 Lc0 v0.21.1-42043 : -29.3 46.0 100 46.0%
10 Lc0 v0.21.1-41965 : -29.3 46.0 100 46.0%
11 Lc0 v0.21.1-41889 : -40.3 44.5 100 44.5%
12 Lc0 v0.21.1-41906 : -55.1 42.5 100 42.5%
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
Re: my Lc0 spreadsheet
Hi All
actually testing 180 games
Stockfish 100419 64 BMI2-x12 vs Lc0 v0.21.1-T40.T8.610
5m+3s , ponder ON.
after 116 games Lc0 is 14 games in front !
+26 =78 -12 from sight of Lc0.
How will it end afte 180 games?
regards, C.K.
actually testing 180 games
Stockfish 100419 64 BMI2-x12 vs Lc0 v0.21.1-T40.T8.610
5m+3s , ponder ON.
after 116 games Lc0 is 14 games in front !
+26 =78 -12 from sight of Lc0.
How will it end afte 180 games?
regards, C.K.
-
- Posts: 565
- Joined: Thu Nov 13, 2014 12:03 pm
Re: my Lc0 spreadsheet
kai.. you now look like robert de niro in taxi driver
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
Re: my Lc0 spreadsheet
after two weeks of break, I started a new test run with nn 42441 which has the highest selplay Elo atm.
last sunday I could win the Engine Masters Tournament on Infinitychess in front of 36 players with a tripple GPU nn 42232 / 91.000 nps.
regards, C.K.
https://docs.google.com/spreadsheets/d/ ... sp=sharing
last sunday I could win the Engine Masters Tournament on Infinitychess in front of 36 players with a tripple GPU nn 42232 / 91.000 nps.
regards, C.K.
https://docs.google.com/spreadsheets/d/ ... sp=sharing