Daniel wrote:I am pretty sure the NN alone can't do it, so the debate was if the policy NN can guide the MCTS search precisely to solve tactics.
The task of the policy head should be to indicate "interesting" moves. I do not see why a NN would not be capable of doing something like that.
For example: I read somewhere (perhaps GCP can confirm if this is true or not) that the way Leela Zero resolves ladders is by making the policy head give high weight to the moves that play out the ladder.
(Note I am neither "for" nor "against" Leela. I see it as a scientific experiment to measure how far the A0 approach will go.)
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
nabildanial wrote:This test should prove that the policy head does improve tactically and you could say that Leela is learning tactics via pattern recognition.
nabildanial wrote:This test should prove that the policy head does improve tactically and you could say that Leela is learning tactics via pattern recognition.
This sheet contains only 1 position.
It needs at least a few hundred to have some significance.
It is a pretty big result considering that older nets than id199 couldn't find the correct move by using only the policy head. Most of us over Discord didn't even believe it would ever happen. I mean the move is absurdly hard to find without search, even for humans. I guess now people can now look for more similar positions for these kind of tests to be more statistically significant.
Vinvin wrote:I ran LCzero, v125 and v188 through the well known WACNEW test suite.
300 tactical positions fairly easy for classical engines.
In 2003, Movei solved 275 on 300 in less than 1 second source : https://www.stmintz.com/ccc/index.php?id=289623
But for LC0, it's another story !
Only 130 positions solved for v125 and 124 solved for v188. Yes, it solves less now than 2 weeks ago !
Conditions : 10 seconds per position (speed around 700 nodes/sec)
With today's network (201) a little bit better here: 155 of 300.
Gave her 15" per move on 24 threads of a 12x3GHz CPU, pity none of my GPUs do work with her, one is AMD and doesn't support OpenCl, the other one doesn't work at all, giving unspecific windows- error- message, btw on a Laptop of which CPU isn't compatibel with LZC neither at all.
But it seems to me, the 24 threads of the CPU are somewhat compareable to an average GPU, showing between 1 and 6 kn/s most of the times, reaching on average depth of 20 at once and up to 22 at these test positions in given time.
Damir wrote:Leela's King Safety and Tactical ability are very weak. . This must be improved.
About the tactics, you're kind of right, at least it regularly still happens against strong engines that they find winning lines against her. About King safety? I highly doubt that, since this is an aspect of positional play, in which Leela excels in general. Just because some testing suite says that she neglects king safety doesn't mean that this actually happens in real play. Leela just tends to use the king very actively once it's reasonably safe to do so...
Damir wrote:Leela's King Safety and Tactical ability are very weak. . This must be improved.
About the tactics, you're kind of right, at least it regularly still happens against strong engines that they find winning lines against her. About King safety? I highly doubt that, since this is an aspect of positional play, in which Leela excels in general. Just because some testing suite says that she neglects king safety doesn't mean that this actually happens in real play. Leela just tends to use the king very actively once it's reasonably safe to do so...
Damir wrote:Leela's King Safety and Tactical ability are very weak. . This must be improved.
I find Leela's understanding of king safety very advanced actually. This is seen not only in her impressive attacking ability, but the evaluations she displays when one king is exposed or stuck in the center.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
Vinvin wrote:I ran LCzero, v125 and v188 through the well known WACNEW test suite.
300 tactical positions fairly easy for classical engines.
In 2003, Movei solved 275 on 300 in less than 1 second source : https://www.stmintz.com/ccc/index.php?id=289623
But for LC0, it's another story !
Only 130 positions solved for v125 and 124 solved for v188. Yes, it solves less now than 2 weeks ago !
[/code]
FWIW, someone on the leela-chess github made the correct observation that all the training is done with the full move history (8 moves IIRC). If you feed it isolated positions, it won't necessarily behave as you expect.
(As a human you'd only use the past positions for a repetition check, but when training a DCNN, good luck figuring out what it's doing...)
