Lc0 51010

[lkaufman]

Finally the real elo is no longer keeping up with the self elo. Network 51080, selfrated 1800, beat Stockfish 10 single-thread by 11 to 9 (+35 elo), which gives a performance about 2490 CEGT blitz, 2530 CCRL blitz. So about +60 elo over the last test for a 100 gain in self-elo. It's a bit puzzling to me that selfplay elo works reasonably well for Stockfish and Komodo; why is it so different for Lc0? Insufficient opening variety perhaps?

[chrisw]

Finally the real elo is no longer keeping up with the self elo. Network 51080, selfrated 1800, beat Stockfish 10 single-thread by 11 to 9 (+35 elo), which gives a performance about 2490 CEGT blitz, 2530 CCRL blitz. So about +60 elo over the last test for a 100 gain in self-elo. It's a bit puzzling to me that selfplay elo works reasonably well for Stockfish and Komodo; why is it so different for Lc0? Insufficient opening variety perhaps?

you seem to have been asking this question and variations of it, over and over in this thread.

Back to basics ... there are two worlds that fundamentally differ, the AB programs and the NN program.
And you are comparing self play between AB-AB with selfplay NN-NN. Both produce self elos.

You then state AB-AB self play elo “works” reasonably well, but NN-NN is “so different”.

Well, the answer is in your description and metric “works”. By works, I guess you mean in accord with some sort of established reality, but this reality is of the AB-AB world. When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

[Werewolf]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

[jp]

AZ showed it.

It isn't clear what AZ showed related to self-elo either.
It's easier for Lc to show something because we can examine it.

[chrisw]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

[lkaufman]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

[chrisw]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

[lkaufman]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this. They are just the best we can do as human players, unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion, but I don't know of any new rules that human players can use as a result of the NNs, except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

[Albert Silver]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this. They are just the best we can do as human players, unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion, but I don't know of any new rules that human players can use as a result of the NNs, except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

I don't know that that is the right question to be asking. I don't agree that zeroism is some holy grail either, but do agree that the NNs are going to be bringing about a number of changes and evolved understanding of chess.Trying to dismiss this as being dependent on some new "rule", without which there is nothing is I think an act of denial. A/B engines have been redefining our understanding of chess for decades, but I cannot think of a single "rule" they brought, other than: argue less with them. The way a wide variety of positions are approached, whether openings, middlegames, or endgame, has changed enormously over the last decades, and there is no question whatsoever on the cause: massive engine analysis. What is playable, what is not, the best way to proceed or not, and the list goes on. NNs will bring about more changes without doubt.

Some years ago, I saw a brilliant collection of games by Kasparov, analyzed by him and GM Plaskett on video, and Kasparov explained that his approach was entirely in dynamic relations. Pawns, exchanges, and pieces were completely interchangeable for weakened kings, pawn structures, advanced pawns and powerful outposts. Sure, this is fairly standard practice, especially nowadays, but few did this quite as liberally as he did. The play displayed by AZ and the NNs in general remind me greatly of this, and I think we will see a gradual shift by the newer generations of players who grow up with NNs, with fewer preconceptions restraining them.

[chrisw]

When SF self plays, it’s not that much different from playing any good AB program, elo-wise. No reason for self-play to be wildly off, other than the usual inflation effects.
But every reason for NN-NN self play to not be in accord with the AB world. You have to adjust, everything you know about chess is wrong. AZ showed it.

This isn't clear.

It’s not clear that AZ (and LZ) showed that everything you know about chess is wrong?!

Really? That’s not clear?

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.

Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.

They are just the best we can do as human players,

nothing to do with computers, these were human heuristics, we’re very good at heuristics.

unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,

that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.

but I don't know of any new rules that human players can use as a result of the NNs,

that’s a continuum assumption. Tear up the old rules and realise that SF is not god.

except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.