Lc0 51010

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Albert Silver
Posts: 2860
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Lc0 51010

Post by Albert Silver » Mon Apr 01, 2019 7:09 pm

chrisw wrote:
Mon Apr 01, 2019 6:58 pm
btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

User avatar
cdani
Posts: 2104
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Lc0 51010

Post by cdani » Mon Apr 01, 2019 10:36 pm

chrisw wrote:
Mon Apr 01, 2019 6:58 pm
lkaufman wrote:
Mon Apr 01, 2019 5:19 pm
chrisw wrote:
Mon Apr 01, 2019 4:56 pm
lkaufman wrote:
Mon Apr 01, 2019 3:54 pm

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.
That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.
Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.
They are just the best we can do as human players,
nothing to do with computers, these were human heuristics, we’re very good at heuristics.
unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,
that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.
but I don't know of any new rules that human players can use as a result of the NNs,
that’s a continuum assumption. Tear up the old rules and realise that SF is not god.
except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?
well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.

lkaufman
Posts: 3722
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Mon Apr 01, 2019 10:51 pm

Albert Silver wrote:
Mon Apr 01, 2019 7:09 pm
chrisw wrote:
Mon Apr 01, 2019 6:58 pm
btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.
I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.
Komodo rules!

jp
Posts: 815
Joined: Mon Apr 23, 2018 5:54 am

Re: Lc0 51010

Post by jp » Mon Apr 01, 2019 11:13 pm

lkaufman wrote:
Mon Apr 01, 2019 10:51 pm
Albert Silver wrote:
Mon Apr 01, 2019 7:09 pm
chrisw wrote:
Mon Apr 01, 2019 6:58 pm
btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.
I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.
How does nodes=1 make Lc play better than nodes=0?

What is the difference in strength of the two settings? For older networks, there's no way nodes=0 could beat a very strong player.

lkaufman
Posts: 3722
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Tue Apr 02, 2019 1:33 am

jp wrote:
Mon Apr 01, 2019 11:13 pm
lkaufman wrote:
Mon Apr 01, 2019 10:51 pm
Albert Silver wrote:
Mon Apr 01, 2019 7:09 pm
chrisw wrote:
Mon Apr 01, 2019 6:58 pm
btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.
I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.
How does nodes=1 make Lc play better than nodes=0?

What is the difference in strength of the two settings? For older networks, there's no way nodes=0 could beat a very strong player.
As far as I can tell, nodes = 0, nodes = 1, and depth = 1 all play the same moves, which appear to be the policy network move, since they all show a zero centipawn score or 50% win prob. consistently. I tried network 11248 at depth 1 and it beat me soundly in a blitz game. When you say it couldn't beat a strong player, did you mean at standard tc or at blitz, and what is the basis for your statement?
Komodo rules!

carldaman
Posts: 1717
Joined: Sat Jun 02, 2012 12:13 am

Re: Lc0 51010

Post by carldaman » Tue Apr 02, 2019 3:06 am

Excuse me for a moment, but where does one set nodes or depth for Leela? I'm looking at the uci options in the GUI, but I must be missing something.

Thanks in advance!

Albert Silver
Posts: 2860
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Lc0 51010

Post by Albert Silver » Tue Apr 02, 2019 4:17 pm

carldaman wrote:
Tue Apr 02, 2019 3:06 am
Excuse me for a moment, but where does one set nodes or depth for Leela? I'm looking at the uci options in the GUI, but I must be missing something.

Thanks in advance!
It is something that must be set in the time control or match settings. The Fritz GUI does not support this, but Arena and Cute Chess do.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Raphexon
Posts: 91
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Lc0 51010

Post by Raphexon » Tue Apr 02, 2019 5:10 pm

cdani wrote:
Mon Apr 01, 2019 10:36 pm
chrisw wrote:
Mon Apr 01, 2019 6:58 pm
lkaufman wrote:
Mon Apr 01, 2019 5:19 pm
chrisw wrote:
Mon Apr 01, 2019 4:56 pm
lkaufman wrote:
Mon Apr 01, 2019 3:54 pm

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.
That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.
Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.
They are just the best we can do as human players,
nothing to do with computers, these were human heuristics, we’re very good at heuristics.
unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,
that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.
but I don't know of any new rules that human players can use as a result of the NNs,
that’s a continuum assumption. Tear up the old rules and realise that SF is not god.
except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?
well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.
Chess also has the added advantage (for A/B) that it's discrete, it's a tactical game.
The strongest Go engines (like CrazyStone) were already using MCTS before Google.
And Go engines don't even need to search to be strong.
https://arxiv.org/abs/1511.06410

I'd be very surprised if a chess engine without search can ever be competitive with decent human players. (Roughly 2200-2250 ELO)
Then again, maybe I'm wrong and Lc0's static evaluation is already strong enough.

Ironically, draughts and checkers like games may also not be very suited for A/B (like Go) but due to their much smaller search space it's less of an issue. In the case of Checkers it wasn't an issue since it's weakly solved.
International Draughts(10x10) was a different story, in 2012 Scharzman beat Maximus. And while Maximus wasn't the strongest checkers engine, it still managed to reach an average depth of 24 ply!
https://jhorssen.home.xs4all.nl/Maximus ... aximus.pdf

24ply is insane because another study claims that Houdini 1.5 at 20 ply is as strong as Magnus.

https://pdfs.semanticscholar.org/047f/6 ... f58666.pdf

Obviously part of the reason why humans may have enjoyed a bigger advantage in draughts is that it never attracted the same attention of programmers that chess did. That's true but the same goes for the playerbase and a smaller playerbase is less likely to have Magnus level talents.

On the other hand, why I think draughts-like games (and Go*) are intrinsically harder for A/B search is that since every piece is the same it's much more of a positional game. And while chess has a bigger search space it also has many more obvious bad moves that are quickly pruned. You may have a queen that can move in 12 different ways but if 11 of those result in an immediate loss of queen (without gain) then it's really just a piece that can make 1 move.
On the other hand, since the search space is so small of draughts like games on 8x8 and 10x10 boards it's effectively "brute-forceable" by now.

*From what I've read that even Go on smaller boards with a smaller search space than chess aren't exactly suited for A/B.

Raphexon
Posts: 91
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Lc0 51010

Post by Raphexon » Tue Apr 02, 2019 5:41 pm

Seems like I spoke too quickly and 0 node Leela is already quite strong at Blitz.

chrisw
Posts: 2087
Joined: Tue Apr 03, 2012 2:28 pm

Re: Lc0 51010

Post by chrisw » Tue Apr 02, 2019 7:15 pm

Raphexon wrote:
Tue Apr 02, 2019 5:10 pm
cdani wrote:
Mon Apr 01, 2019 10:36 pm
chrisw wrote:
Mon Apr 01, 2019 6:58 pm
lkaufman wrote:
Mon Apr 01, 2019 5:19 pm
chrisw wrote:
Mon Apr 01, 2019 4:56 pm
lkaufman wrote:
Mon Apr 01, 2019 3:54 pm

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.
That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.
Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.
Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.
They are just the best we can do as human players,
nothing to do with computers, these were human heuristics, we’re very good at heuristics.
unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,
that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.
but I don't know of any new rules that human players can use as a result of the NNs,
that’s a continuum assumption. Tear up the old rules and realise that SF is not god.
except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?
well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.
The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.
Chess also has the added advantage (for A/B) that it's discrete, it's a tactical game.
The strongest Go engines (like CrazyStone) were already using MCTS before Google.
And Go engines don't even need to search to be strong.
https://arxiv.org/abs/1511.06410

I'd be very surprised if a chess engine without search can ever be competitive with decent human players. (Roughly 2200-2250 ELO)
Then again, maybe I'm wrong and Lc0's static evaluation is already strong enough.

Ironically, draughts and checkers like games may also not be very suited for A/B (like Go) but due to their much smaller search space it's less of an issue. In the case of Checkers it wasn't an issue since it's weakly solved.
International Draughts(10x10) was a different story, in 2012 Scharzman beat Maximus. And while Maximus wasn't the strongest checkers engine, it still managed to reach an average depth of 24 ply!
https://jhorssen.home.xs4all.nl/Maximus ... aximus.pdf

24ply is insane because another study claims that Houdini 1.5 at 20 ply is as strong as Magnus.

https://pdfs.semanticscholar.org/047f/6 ... f58666.pdf

Obviously part of the reason why humans may have enjoyed a bigger advantage in draughts is that it never attracted the same attention of programmers that chess did. That's true but the same goes for the playerbase and a smaller playerbase is less likely to have Magnus level talents.

On the other hand, why I think draughts-like games (and Go*) are intrinsically harder for A/B search is that since every piece is the same it's much more of a positional game. And while chess has a bigger search space it also has many more obvious bad moves that are quickly pruned. You may have a queen that can move in 12 different ways but if 11 of those result in an immediate loss of queen (without gain) then it's really just a piece that can make 1 move.
On the other hand, since the search space is so small of draughts like games on 8x8 and 10x10 boards it's effectively "brute-forceable" by now.

*From what I've read that even Go on smaller boards with a smaller search space than chess aren't exactly suited for A/B.
There was always going to be a game whose rule system put it at the edge of what is and isn’t solvable. Add in a bunch of emotional reasons, even Soviet USA politicing, the fact that it’s the game played in the dominant white man culture, and that game happens to have been chess, white European version, and it stayed somehow as the main game, when others could well have taken over. Peak chess has been and gone, probably taking with it the other contenders too. I’m inclined to take the position it was all much overrated and we didn’t learn anything much generalisable elsewhere once the basic search algorithm was worked out. Neural nets can make some sort of visual order out of the chess board space, but again, so what?, making visual order out of real world is the thing, not chess boards. So long and thanks for all the fish.

Post Reply