Lc0 51010
Moderators: hgm, Rebel, chrisw
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Lc0 51010
Finally the real elo is no longer keeping up with the self elo. Network 51080, selfrated 1800, beat Stockfish 10 single-thread by 11 to 9 (+35 elo), which gives a performance about 2490 CEGT blitz, 2530 CCRL blitz. So about +60 elo over the last test for a 100 gain in self-elo. It's a bit puzzling to me that selfplay elo works reasonably well for Stockfish and Komodo; why is it so different for Lc0? Insufficient opening variety perhaps?
Komodo rules!
-
- Posts: 4313
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Lc0 51010
you seem to have been asking this question and variations of it, over and over in this thread.lkaufman wrote: ↑Mon Apr 01, 2019 6:56 am Finally the real elo is no longer keeping up with the self elo. Network 51080, selfrated 1800, beat Stockfish 10 single-thread by 11 to 9 (+35 elo), which gives a performance about 2490 CEGT blitz, 2530 CCRL blitz. So about +60 elo over the last test for a 100 gain in self-elo. It's a bit puzzling to me that selfplay elo works reasonably well for Stockfish and Komodo; why is it so different for Lc0? Insufficient opening variety perhaps?
Back to basics ... there are two worlds that fundamentally differ, the AB programs and the NN program.
And you are comparing self play between AB-AB with selfplay NN-NN. Both produce self elos.
You then state AB-AB self play elo “works” reasonably well, but NN-NN is “so different”.
Well, the answer is in your description and metric “works”. By works, I guess you mean in accord with some sort of established reality, but this reality is of the AB-AB world. When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
-
- Posts: 1796
- Joined: Thu Sep 18, 2008 10:24 pm
Re: Lc0 51010
This isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
-
- Posts: 1470
- Joined: Mon Apr 23, 2018 7:54 am
-
- Posts: 4313
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Lc0 51010
It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Lc0 51010
What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.chrisw wrote: ↑Mon Apr 01, 2019 2:02 pmIt’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
Komodo rules!
-
- Posts: 4313
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Lc0 51010
That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.lkaufman wrote: ↑Mon Apr 01, 2019 5:54 pmWhat AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.chrisw wrote: ↑Mon Apr 01, 2019 2:02 pmIt’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Lc0 51010
Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this. They are just the best we can do as human players, unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion, but I don't know of any new rules that human players can use as a result of the NNs, except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?chrisw wrote: ↑Mon Apr 01, 2019 6:56 pmThat would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.lkaufman wrote: ↑Mon Apr 01, 2019 5:54 pmWhat AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.chrisw wrote: ↑Mon Apr 01, 2019 2:02 pmIt’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
Komodo rules!
-
- Posts: 3019
- Joined: Wed Mar 08, 2006 9:57 pm
- Location: Rio de Janeiro, Brazil
Re: Lc0 51010
I don't know that that is the right question to be asking. I don't agree that zeroism is some holy grail either, but do agree that the NNs are going to be bringing about a number of changes and evolved understanding of chess.Trying to dismiss this as being dependent on some new "rule", without which there is nothing is I think an act of denial. A/B engines have been redefining our understanding of chess for decades, but I cannot think of a single "rule" they brought, other than: argue less with them. The way a wide variety of positions are approached, whether openings, middlegames, or endgame, has changed enormously over the last decades, and there is no question whatsoever on the cause: massive engine analysis. What is playable, what is not, the best way to proceed or not, and the list goes on. NNs will bring about more changes without doubt.lkaufman wrote: ↑Mon Apr 01, 2019 7:19 pmOf course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this. They are just the best we can do as human players, unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion, but I don't know of any new rules that human players can use as a result of the NNs, except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?chrisw wrote: ↑Mon Apr 01, 2019 6:56 pmThat would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.lkaufman wrote: ↑Mon Apr 01, 2019 5:54 pmWhat AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.chrisw wrote: ↑Mon Apr 01, 2019 2:02 pmIt’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
Some years ago, I saw a brilliant collection of games by Kasparov, analyzed by him and GM Plaskett on video, and Kasparov explained that his approach was entirely in dynamic relations. Pawns, exchanges, and pieces were completely interchangeable for weakened kings, pawn structures, advanced pawns and powerful outposts. Sure, this is fairly standard practice, especially nowadays, but few did this quite as liberally as he did. The play displayed by AZ and the NNs in general remind me greatly of this, and I think we will see a gradual shift by the newer generations of players who grow up with NNs, with fewer preconceptions restraining them.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
-
- Posts: 4313
- Joined: Tue Apr 03, 2012 4:28 pm
Re: Lc0 51010
Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.lkaufman wrote: ↑Mon Apr 01, 2019 7:19 pmOf course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.chrisw wrote: ↑Mon Apr 01, 2019 6:56 pmThat would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.lkaufman wrote: ↑Mon Apr 01, 2019 5:54 pmWhat AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.chrisw wrote: ↑Mon Apr 01, 2019 2:02 pmIt’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!Werewolf wrote: ↑Mon Apr 01, 2019 10:25 amThis isn't clear.chrisw wrote: ↑Mon Apr 01, 2019 10:14 am When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.
Really? That’s not clear?
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
nothing to do with computers, these were human heuristics, we’re very good at heuristics.They are just the best we can do as human players,
that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,
that’s a continuum assumption. Tear up the old rules and realise that SF is not god.but I don't know of any new rules that human players can use as a result of the NNs,
well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?