Page 1 of 1
MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 1:32 am
by brtzsnr
Consider this position
https://lichess.org/KOImYPTl#60 which is highly favorable for white.
[D] R7/2B4p/1p2k1p1/8/1Pp3p1/2Q3P1/1P2PP1P/5RK1 w - - 6 31
The game is between two instances of zurichess mcts (less_than_one). Once the game reaches this extreme imbalance the engine moves aimlessly because all moves are completely winning for white and completely losing for black. White doesn't care about dropping the queen and then a rook since it is still winning.
What are good ways to improve the engine to make some real progress in this type of positions?
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 8:13 am
by Ferdy
brtzsnr wrote: ↑Wed Jan 16, 2019 1:32 am
What are good ways to improve the engine to make some real progress in this type of positions?
While you may have
Code: Select all
move_score = Exploit() + Explore()
where:
Code: Select all
Exploit = f(winrate)
Explore = f(visits)
Perhaps you may use
so that moves that gives material will be discouraged
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 8:29 am
by Uri Blass
I think that the best way is to change the scoring system so mate in 200 moves is worse than mate in 100 moves.
For example winning give the winning side 1-0.0001*number of moves and the losing side 0.0001*number of moves.
In this case mate in 100 gives 0.99 points and mate in 200 gives 0.98 points.
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 8:45 am
by hgm
Yeah, I already raised that point in a cute-chess issue discussion as well. The purpose of the heuristic evaluation is not only to reflect winning probability, but also to reflect progress. It is very bad to let it saturate too easily. Centi-Pawn scoring doesn't suffer from such saturation, and remains sensitive to small positional terms like centralisation no matter how large the advantage. But compressing the scale pushes such contributions below the noise level.
I guess you can only make it right by training it properly, e.g. like Uri suggests. Not only on the results of the games, but also on their duration.
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 10:25 am
by chrisw
brtzsnr wrote: ↑Wed Jan 16, 2019 1:32 am
Consider this position
https://lichess.org/KOImYPTl#60 which is highly favorable for white.
[D] R7/2B4p/1p2k1p1/8/1Pp3p1/2Q3P1/1P2PP1P/5RK1 w - - 6 31
The game is between two instances of zurichess mcts (less_than_one). Once the game reaches this extreme imbalance the engine moves aimlessly because all moves are completely winning for white and completely losing for black. White doesn't care about dropping the queen and then a rook since it is still winning.
What are good ways to improve the engine to make some real progress in this type of positions?
Fold depth to win into the evaluation
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 5:02 pm
by brtzsnr
Uri Blass wrote: ↑Wed Jan 16, 2019 8:29 am
I think that the best way is to change the scoring system so mate in 200 moves is worse than mate in 100 moves.
For example winning give the winning side 1-0.0001*number of moves and the losing side 0.0001*number of moves.
In this case mate in 100 gives 0.99 points and mate in 200 gives 0.98 points.
This is in interesting idea. I suspect that it'll give kingsafety a larger score. It doesn't pass yet my tests. I tried: 1e-4 -> -40Elo. 5e-5 -> +2Elo, 5e-5 -> +3Elo. I will try again with higher coefficients.
What worked for about 30Elo is to multiply the score by 0.75 before sigmoid. Instead of sigmoid(evaluateCP(pos) * 1e-2) now I have sigmoid(evaluateCP(pos) * 1e-2 * 0.75).
Re: MCTS: How to deal with extreme imbalances
Posted: Wed Jan 16, 2019 6:45 pm
by Daniel Shawul
brtzsnr wrote: ↑Wed Jan 16, 2019 1:32 am
Consider this position
https://lichess.org/KOImYPTl#60 which is highly favorable for white.
[D] R7/2B4p/1p2k1p1/8/1Pp3p1/2Q3P1/1P2PP1P/5RK1 w - - 6 31
The game is between two instances of zurichess mcts (less_than_one). Once the game reaches this extreme imbalance the engine moves aimlessly because all moves are completely winning for white and completely losing for black. White doesn't care about dropping the queen and then a rook since it is still winning.
What are good ways to improve the engine to make some real progress in this type of positions?
Picking the move with the maximum score instead of the with most number of visits helps.
I also do a MATE_SCORE - ply correction when mating. This is possible because i use raw scores not fractions (0...1) for mcts bookeeping