LCZero: Progress and Scaling. Relation to CCRL Elo

[yanquis1972]

albert, did you have smart pruning checked or unchecked? have you tried significantly longer time permitted (2x or more) to see if there's a substantial difference?

[Nay Lin Tun]

This is tactics, if you see a variation with one piece advantage, but there is a better variation that leads one rook advantage, you fail.
You see mate in 10, but there is mate in 6, you fail.
This is general principal of tactics in " chess tempo, lichess and chess.com"

"You are asked to show the best move, not asked to proove which side will win"

[yanquis1972]

is there evidence from the released alphazero games the tactics will ever reach that level? basically, if it determined a 99%+ chance of winning, did it strive for 100%? or might it have learned it was irrelevant?

[Albert Silver]

This is tactics, if you see a variation with one piece advantage, but there is a better variation that leads one rook advantage, you fail.
You see mate in 10, but there is mate in 6, you fail.
This is general principal of tactics in " chess tempo, lichess and chess.com"

"You are asked to show the best move, not asked to proove which side will win"

Perfectionism is great, but not at the cost of practicality. Training to find the best move is essential, since in many cases there is no forgiveness. However, with time, I have come to view the middlegame much like the endgame, in that if a won position has been reached, so long as it is won, and never squandered, I will not lose sleep over a move-count.

How often do we read about choosing the path of least resistance even if the most 'precise' move, which would allow all sorts of dangerous possibilities, is discarded?

[Albert Silver]

albert, did you have smart pruning checked or unchecked? have you tried significantly longer time permitted (2x or more) to see if there's a substantial difference?

I am using the latest May 9 build, and smart pruning is checked. The only changes I make are cPUCT MCTS=3.0, the FPU is reset to 0.0, and the time scaling is 2.0 for actual games. I strongly disagree with anything lower and faster.

[peter]

Ok, so I tested it with 10 seconds per move, which is what i understood you do, and got:

Analysis ended normally 164 of 300 matching moves Rated time: 24:30

...

Don't get me wrong, I am not suggesting Leela is any stronger in tactics because of this, but this test does little to prove anything in my eyes. At the very least it would need some serious house-cleaning to remove all the positions with multiple solutions.

Thanks for trying, Albert, that's what I was interested in, just because of course I don't have any idea about the relation of my 24 threaded CPU- version against a really good graphic card. n/s and dephts (rollouts?) to me seem comparable to a mediocre graphic card, and ten seconds is just really much for WAC.
Yet 164 is some better then 150 or even only 140 latest. And your result yet might show hardware to be even more important as for tactics than as for "positional play".
Compared to other engines 164/300 is just bad still, isn't it`?

With one second per move on single thread, most modern engines solve all or alomost all the positions, even if there are some doubtable positions among them.

And of course you're right, the result of any single (tactical or positional) best move test suite always has to be seen as a single test result only.
But the reason, I repeatedly took WAC was, that Vincent Lejeune gave his LC0- results of this one some time ago here, and I simply couldn't find any other tactical test suite more easy for modern engines with yet quite a number of not so bad best move- positions.

At 300 there have to be always some with doubtable single best move as for being the one and only single one for sure, but that if there may be other winning moves in winning positions too, that's a problem of its own as for test positions commonly.

Question to be answered always here has to be only, is the move given as solution really the best one. Do you doubt the two examples you gave as for this one only relevant question? A mate in two still is better then any other move in the given position, that doesn't mate in two or in one.


And being too puristic about that, you just don't get 300 easily solvable tactical test positions of some level they have to have in common, considering, they must not be much to easy neither.

The next one suite I thought about trying would be chessbase- "Marathon", not so difficult neither, but I simply don't see LC0 have any discussable results with this one at 10"/move, maybe I yet will give it a try with some even longer TC, if I find the time for it.

For my personal pov, I don't need tactical test position suites at all, seeing output- lines of an engine at single positions of interest, with or without undoubtable single best move, tells me more about the tactical and "positional" abilities of an engine then suites ever can, but of course always only as for the single one position in question.

And then there is always the old argument of statistics- lovers, "single positions don't count at all", that's nonsense too of course, chess consists of single positions and single moves only always. Game results without knowing the games and the moves one by one don't count per se at all neither, not even statistically, if you don't relativize the results as for TC, hardware (GPU- CPU in this case new), and e.g. for this one engine of special importance too: as for the opponents and openings chosen.

Have you already found a better (easier) test suite with tactical best move positions solveable for LC0?

[Nay Lin Tun]

This is tactics, if you see a variation with one piece advantage, but there is a better variation that leads one rook advantage, you fail.
You see mate in 10, but there is mate in 6, you fail.
This is general principal of tactics in " chess tempo, lichess and chess.com"

"You are asked to show the best move, not asked to proove which side will win"

Perfectionism is great, but not at the cost of practicality. Training to find the best move is essential, since in many cases there is no forgiveness. However, with time, I have come to view the middlegame much like the endgame, in that if a won position has been reached, so long as it is won, and never squandered, I will not lose sleep over a move-count.

How often do we read about choosing the path of least resistance even if the most 'precise' move, which would allow all sorts of dangerous possibilities, is discarded?

I agreed your point. The main goal of chess is to win safely. That is why good players usually do solid and simple pathway to win and avoid tactics as much as possible. ( e.g trading rook vs minor , if that lead to solid pawns endgames).

On the other side, if you are doing tactics and puzzles, the goal is finding the shortest and most effective move.

[Laskos]

This is tactics, if you see a variation with one piece advantage, but there is a better variation that leads one rook advantage, you fail.
You see mate in 10, but there is mate in 6, you fail.
This is general principal of tactics in " chess tempo, lichess and chess.com"

"You are asked to show the best move, not asked to proove which side will win"

Perfectionism is great, but not at the cost of practicality. Training to find the best move is essential, since in many cases there is no forgiveness. However, with time, I have come to view the middlegame much like the endgame, in that if a won position has been reached, so long as it is won, and never squandered, I will not lose sleep over a move-count.

How often do we read about choosing the path of least resistance even if the most 'precise' move, which would allow all sorts of dangerous possibilities, is discarded?

Yes, I agree, but there is a problem. Maybe WAC contains many of such "multiple solutions to the same outcome", but from what I get they are not the majority. Take this result I got earlier in this thread:

==========
It was interesting to see LC0 trashing an AB engine in normal games from normal, balanced starting positions:

Openings: 3moves_GM.epd (side and reversed)
Score of LC0_245 vs Predateur 2.2.1: 93 - 1 - 6 [0.960] 100
ELO difference: 552.08 +/- 173.83
Finished match

But from WAC starting position (side and reversed), so in games having 1 tactical shot, it manages to lose to an engine considered in normal (above) conditions almost 600 Elo points weaker:

Openings: WAC300.epd (side and reversed)
Score of LC0_245 vs Predateur 2.2.1: 42 - 52 - 6 [0.450] 100
ELO difference: -34.86 +/- 66.75
Finished match
===========

So, it's not only the move count, it's also the outcome of the games which is changed in these WAC positions, if not solved properly and uniquely. At least in the majority of WAC puzzles.

[jp]

"You are asked to show the best move, not asked to proove which side will win"

Perfectionism is great, but not at the cost of practicality. Training to find the best move is essential, since in many cases there is no forgiveness. However, with time, I have come to view the middlegame much like the endgame, in that if a won position has been reached, so long as it is won, and never squandered, I will not lose sleep over a move-count.

How often do we read about choosing the path of least resistance even if the most 'precise' move, which would allow all sorts of dangerous possibilities, is discarded?

Yes, I agree, but there is a problem. Maybe WAC contains many of such "multiple solutions to the same outcome", but from what I get they are not the majority. Take this result I got earlier in this thread:

Yes. Kai's result shows WAC is not the problem.

Nay Lin Tun & Albert talk like LC0 is human. It's irrelevant talking about choosing the path of least resistance, etc., which a human deliberately does for practical reasons when he knows there's a 'better' move. LC0 won't do that. LC0 just doesn't know there's a 'better' move.

[Albert Silver]

Perfectionism is great, but not at the cost of practicality. Training to find the best move is essential, since in many cases there is no forgiveness. However, with time, I have come to view the middlegame much like the endgame, in that if a won position has been reached, so long as it is won, and never squandered, I will not lose sleep over a move-count.

How often do we read about choosing the path of least resistance even if the most 'precise' move, which would allow all sorts of dangerous possibilities, is discarded?

Yes, I agree, but there is a problem. Maybe WAC contains many of such "multiple solutions to the same outcome", but from what I get they are not the majority. Take this result I got earlier in this thread:

Yes. Kai's result shows WAC is not the problem.

Nay Lin Tun & Albert talk like LC0 is human. It's irrelevant talking about choosing the path of least resistance, etc., which a human deliberately does for practical reasons when he knows there's a 'better' move. LC0 won't do that. LC0 just doesn't know there's a 'better' move.

I think you need to reread what I wrote since I said nothing of the sort.