pferd wrote:This seems like a very interesting project.
I am playing some 5 minute games against it right now and it moves instantly every single time. Is this the expected behaviour?
Looking at the games it seems that something is wrong.
Google got level above GM strength.
I understand that they had hardware advantage but still
I think that it should be possible to get at least above level of 2000 by similiar methods and the level that I see that is simply stupid mistakes of losing pieces is not even close.
Beating the old version by result that is not close to 100-0 is too slow progress.
Google got something that get better if you give it more time.
If I understand correctly lczero does not get better with more time because it play immediately when time control is not relevant.
The whole point of this initial supervised learning run is to validate that the search is working well. It's still very early in the learning phases. Google played many orders of magnitude more games to get the results they did.
Things are looking good so far, but yes, it's still making huge blunders. We will see if it learns it's way out of them though .
pferd wrote:This seems like a very interesting project.
I am playing some 5 minute games against it right now and it moves instantly every single time. Is this the expected behaviour?
This is because the default number of playouts is set to 800, and there is no time management.
You can increase the number of playouts by passing the -p argument on the command line, like -p20000, and the engine will think for longer, and play stronger.
Thanks. That helped improving the playing strength quite a bit. Using -p20000 --noponder made the engine think for approximately 5s/move on my machine.
It still offers its queen every once in a while though
pferd wrote:This seems like a very interesting project.
I am playing some 5 minute games against it right now and it moves instantly every single time. Is this the expected behaviour?
This is because the default number of playouts is set to 800, and there is no time management.
You can increase the number of playouts by passing the -p argument on the command line, like -p20000, and the engine will think for longer, and play stronger.
Thanks. That helped improving the playing strength quite a bit. Using -p20000 --noponder made the engine think for approximately 5s/move on my machine.
It still offers its queen every once in a while though
This is the inherent tactical issues MCTS have -- which I am very eager to see how the NN could help eliminate it. Not offering your queen requires only a 2-ply lookahead knowledge which the mcts will solve with more simulations but resolving a 4 to 7 plies deep tactics needs tremendous amount of simulations compared to a brute forcer (maybe a 100x more effort is required). Lczero will not be competitive with alpha-beta engines until this issue is solved, lets see if more traiining or bigger network will help in this regard. Using a regular quiescence search instead of the NN, scorpioMCTS does not blunder pieces anymore but can fail to a 7-ply or more tactics -- it is about 100 elo stronger than TSCP.
In the eighties, when there was no hope to do a brute-force search of more than 4 ply, I used an extension scheme to see deeper tactics that tried to follow the lines that consisted of logical combinations of moves. That is, it would extend all captures that were new compared to the previous position with that side to move (e.g. recapture, moves over the square evacuated on the previous ply ('pins') and the ply before that ('discovered attacks')) by a full ply, and similar non-captures by a quarter ply. This worked pretty well: sacrifices that were pointless did not enable any follow-up moves, and thus could be refuted with small trees. But for sacrifices that would enable other stuff that other stuff would get extended, and if it would indeed lead to some return on investment, you would see it.
It seems to me that the MCTS move selection would need something similar. Note that the AlphaZero NN doesn't only get the current position as input, but also a few positions before it. This is necessary to recognize repettions. But it can also use it to learn what logical follow-ups are that often (but not always) give good results. E.g. capturing a protected Bishop with a Rook is a bad idea. But when the protector is soft-pinned (i.e. there is a follow-up capture over the square evacuated by the re-capture) it should be considered wth high priority. A NN should not have much problems recogizing such patterns, and recommending the move combinations.
Do you think an MCTS engine which can look ahead to a fixed depth seach at every ply ( say a 7-ply lookahead) can be come competitive with a selective alpha-beta engine that can lookahead 20-plies ahead from the root ? This is the only way I see how it can beat an alpha-beta engine as it seems to me that an MCTS engine can not become tactically stronger at the same rate as the alpha-beta engine. Sure it can begin to realize shallow tactics with more simulation time but the deeper tactics (4 to 7-ply) require a lot more to recongnize.
Here is an example game where I increased the number of simulations to p=15000 for lczero. It takes about 60 secs per move on my intel hd graphics card (i guess very slow). So many blunders in there, e.g. look at the ending where it fails to find a mate-in-2 tactics. Note that the only difference b/n ScorpioMCTS and Lczero is i use qsearch instead of the NN -- which I allows me to do far more playouts/sec than lczero and hence resolve more shallow tactics by growing the tree faster. So far the policy NN is not helping in resolving tactics
I have no experience with MCTS, but since you seem to need so many nodes to see so little, I wonder if it makes proper use of beta cutoffs. E.g. in a hypothetical situation where you have the choice between two moves, one somewhat better than the other, while after that it doesn't matter much what you play for many moves in the case where you chose the weaker move (i.e. a no-progress situation). The UCT algorithm would channel more playouts into the better move, but if the difference is only small, it would be forced to keep exploring the weaker move as well, to make sure the upper confidence bound of that stays below the one of the best move.
Now if I understand MCTS correctly, all visits of the weaker moves would homogeneously sample the sub-tree behind it, as there is no value difference there. So it will converge to a minimax tree. Which seems very wrong, as it would be good eough to have a refutation tree. If all these visits would refrain from exploring any other moves as long as the upper confidence bound of the one move they explore stays below beta (for which you could take the lower confidence bound of the good initial move), it just should keep exploiting that move. MCTS-UCT is modeled on the k-armed-bandit problem, (applying it to every node), where you want to make sure you exploit the best-paying bandit. But it should really try to find a bandit that is good enough.
Usually the k-armed-bandit problem is formulated with arms that have a benign distribution, like Gaussians with fixed standard deviation, only differing in expectation value. Unfortunately random sampling of a Chess tree is not like that at all. It is more like dealing with bandits where most of the contribution to the average comes from a few outcome with very small probablilty but huge payout. You really have no idea of the average until you have enough visits to representatively sample the unlikely events. Likewise, in Chess you will only get a meaningful value when you have enough visits to discover the tactics that define the main line, and start exploiting that.
Usually the k-armed-bandit problem is formulated with arms that have a benign distribution, like Gaussians with fixed standard deviation, only differing in expectation value. Unfortunately random sampling of a Chess tree is not like that at all. It is more like dealing with bandits where most of the contribution to the average comes from a few outcome with very small probablilty but huge payout. You really have no idea of the average until you have enough visits to representatively sample the unlikely events. Likewise, in Chess you will only get a meaningful value when you have enough visits to discover the tactics that define the main line, and start exploiting that.
That is why I do not use averaging as a backup operator for MCTS in chess -- I return the minimax value of the best bandit instead. This gives a more stable search and i think is better tactically as well. A typical example is a position where you are threatening a mate-in-1 but the opponent has one or two escape move that keeps the score at 0. With averaging as a backup operator, the engine would hugely prefer these position as it can get mate scores from all the other non-escape moves. Since Mate scores are very high , even one of them ruins the actual score. My engine with this backup operator exhibits a wildly varying score from move to move. However, with minimax as a backup operator, the engine will keep its score at 0, and the score remains stable like a standard alpha-beta search. Thus, the averaging backup operator is preferred as the better planning method for games dominated with strategy, and the minimax operator for tactical games like chess. Note that the tactical gain i get from using the minimax operator, though significant, is miniscule compared to that from expanding the tree rapidly every visit.
I have no idea how they removed the tactical blindness from A0 -- I am currently stuck at TSCP + 100 elo with the MCTS + qsearch approach.
Usually the k-armed-bandit problem is formulated with arms that have a benign distribution, like Gaussians with fixed standard deviation, only differing in expectation value. Unfortunately random sampling of a Chess tree is not like that at all. It is more like dealing with bandits where most of the contribution to the average comes from a few outcome with very small probablilty but huge payout. You really have no idea of the average until you have enough visits to representatively sample the unlikely events. Likewise, in Chess you will only get a meaningful value when you have enough visits to discover the tactics that define the main line, and start exploiting that.
That is why I do not use averaging as a backup operator for MCTS in chess -- I return the minimax value of the best bandit instead. This gives a more stable search and i think is better tactically as well. A typical example is a position where you are threatening a mate-in-1 but the opponent has one or two escape move that keeps the score at 0. With averaging as a backup operator, the engine would hugely prefer these position as it can get mate scores from all the other non-escape moves. Since Mate scores are very high , even one of them ruins the actual score. My engine with this backup operator exhibits a wildly varying score from move to move. However, with minimax as a backup operator, the engine will keep its score at 0, and the score remains stable like a standard alpha-beta search. Thus, the averaging backup operator is preferred as the better planning method for games dominated with strategy, and the minimax operator for tactical games like chess. Note that the tactical gain i get from using the minimax operator, though significant, is miniscule compared to that from expanding the tree rapidly every visit.
I have no idea how they removed the tactical blindness from A0 -- I am currently stuck at TSCP + 100 elo with the MCTS + qsearch approach.
You are obviously much more an expert than I. But I guess the idea is that an escapable mate threat should not be scored as zero (the minimax score) since it is still has a significant value as a threat (restricting the opponent), and moreover there could be nearby lines where the near mate is actually a mate.
BTW. I assume before applying the averaging operator you first convert scores to a winning probability? It seems obvious but I just wanted to make sure.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Daniel Shawul wrote:Do you think an MCTS engine which can look ahead to a fixed depth seach at every ply ( say a 7-ply lookahead) can be come competitive with a selective alpha-beta engine that can lookahead 20-plies ahead from the root ? This is the only way I see how it can beat an alpha-beta engine as it seems to me that an MCTS engine can not become tactically stronger at the same rate as the alpha-beta engine. Sure it can begin to realize shallow tactics with more simulation time but the deeper tactics (4 to 7-ply) require a lot more to recongnize.
Here is an example game where I increased the number of simulations to p=15000 for lczero. It takes about 60 secs per move on my intel hd graphics card (i guess very slow). So many blunders in there, e.g. look at the ending where it fails to find a mate-in-2 tactics. Note that the only difference b/n ScorpioMCTS and Lczero is i use qsearch instead of the NN -- which I allows me to do far more playouts/sec than lczero and hence resolve more shallow tactics by growing the tree faster. So far the policy NN is not helping in resolving tactics
Agreed, the policy network is still not anywhere close to resolving significant tactics. My current thinking is this is due to the training on SF games. It hasn't had a chance to experience bad moves, and so hasn't learned to refute them.
For your setup though, the CPU version would probably be far faster (just comment out #define USE_OPENCL in config.h). Those Intel GPUs are really bad for NN evals.
.
