Engine losing due to tactical positions

[Witek]

Hi,

I was trying to improve my engine strength by analyzing why it's losing games against other engines of similar strength. This way I already found two major bugs and fixed them. Now there are two things left: slowly loosing which is probably caused by imperfect evaluation and "blundering" in complex tactical positions.

For example, in this position Qb3 is best, which is pretty obvious even for a human, because the queen can be trapped. Caissa finds it in 3 seconds, but Stockfish spots it in around 0.15 seconds.
[fen]r2q1rk1/1Q2npp1/p1p1b2p/b2p4/2nP4/2N1PNP1/PP1B1PBP/R4RK1 w - - 0 17[/fen]


Another example. a5 is best, Caissa finds it in 24 seconds, but Stockfish needs only 0.2 seconds!
[fen]8/2k5/p3PK2/1p6/1Pn2P2/8/5p2/5N2 b - - 6 66[/fen]


I have few other cases like this, and could find more.

And now I'm not sure if it's a bug or the timings to find best move are reasonable. I checked couple other engines on these positions and some of them find best move as quickly as Stockfish and some of them are as slow as my engine...

Is there any sense in building up a list of such positions and trying to optimize the engine to solve them quicker?

[Mike Sherwin]

Hi,

I was trying to improve my engine strength by analyzing why it's losing games against other engines of similar strength. This way I already found two major bugs and fixed them. Now there are two things left: slowly loosing which is probably caused by imperfect evaluation and "blundering" in complex tactical positions.

For example, in this position Qb3 is best, which is pretty obvious even for a human, because the queen can be trapped. Caissa finds it in 3 seconds, but Stockfish spots it in around 0.15 seconds.
[fen]r2q1rk1/1Q2npp1/p1p1b2p/b2p4/2nP4/2N1PNP1/PP1B1PBP/R4RK1 w - - 0 17[/fen]


Another example. a5 is best, Caissa finds it in 24 seconds, but Stockfish needs only 0.2 seconds!
[fen]8/2k5/p3PK2/1p6/1Pn2P2/8/5p2/5N2 b - - 6 66[/fen]


I have few other cases like this, and could find more.

And now I'm not sure if it's a bug or the timings to find best move are reasonable. I checked couple other engines on these positions and some of them find best move as quickly as Stockfish and some of them are as slow as my engine...

Is there any sense in building up a list of such positions and trying to optimize the engine to solve them quicker?

It is a combination of an evaluation that is not as good and reductions. The reductions may not be sound if the evaluation is allowing good moves to be reduced. RomiChess takes 10 seconds to find b7b3 at depth 16. The important metric is not time but depth. SF14 finds b7b3 by depth 11.

[Sopel]

The important metric is not time but depth.

No, depth is the least important metric in most modern engines. It's an implemnetation detail that changes chaoticly from version to version. It holds no value for the user and is only used to provide periodic updates for the UCI.

[Mike Sherwin]

The important metric is not time but depth.

No, depth is the least important metric in most modern engines. It's an implemnetation detail that changes chaoticly from version to version. It holds no value for the user and is only used to provide periodic updates for the UCI.

No, LMR & Null Move reductions greatly affect at which depth a tactic or just the best move will be seen. And the eval greatly affects the order in which the moves will be searched. A weak eval will consistently result in good moves being reduced by LMR thus delaying finding the best move till a deeper ply. In this case it has everything to do with depth and nothing directly to do with time which becomes no more than an artifact of depth. Are you an engine author? Do you even know what LMR is?

[Sopel]

The important metric is not time but depth.

No, depth is the least important metric in most modern engines. It's an implemnetation detail that changes chaoticly from version to version. It holds no value for the user and is only used to provide periodic updates for the UCI.

No, LMR & Null Move reductions greatly affect at which depth a tactic or just the best move will be seen. And the eval greatly affects the order in which the moves will be searched. A weak eval will consistently result in good moves being reduced by LMR thus delaying finding the best move till a deeper ply. In this case it has everything to do with depth and nothing directly to do with time which becomes no more than an artifact of depth. Are you an engine author? Do you even know what LMR is?

You're contradicting yourself. You explicitly write "LMR & Null Move reductions greatly affect at which depth a tactic or just the best move will be seen.", yet still say "In this case it has everything to do with depth".

[Witek]

Agree with Sopel. You can have an engine with very agressive LMR vs. one without it with simillar Elo. The former will be achieving much higher depths, but it doesn't mean that it's stronger or weaker...

Also, saying that evaluation has great impact on move order is partially true. It will determine what the first move to search will be, but the order of remaining moves mostly relies on other heuristics.

[RubiChess]

Are you an engine author? Do you even know what LMR is?

https://www.chessprogramming.org/Tomasz_Sobczyk

[Mike Sherwin]

Agree with Sopel. You can have an engine with very agressive LMR vs. one without it with simillar Elo. The former will be achieving much higher depths, but it doesn't mean that it's stronger or weaker...

Also, saying that evaluation has great impact on move order is partially true. It will determine what the first move to search will be, but the order of remaining moves mostly relies on other heuristics.

And if the first move searched is not best then the real best move can end up being reduced by LMR. Hence it may take more ply to find the best move. You mention aggressive and that is a new consideration. A weak evaluation will work less well the more aggressive the LMR is. I'm done here because Sopal's argument was time versus depth and you do not even address the argument.

[Vernon Crawford]

Agree with Sopel. You can have an engine with very agressive LMR vs. one without it with simillar Elo. The former will be achieving much higher depths, but it doesn't mean that it's stronger or weaker...

Also, saying that evaluation has great impact on move order is partially true. It will determine what the first move to search will be, but the order of remaining moves mostly relies on other heuristics.

And if the first move searched is not best then the real best move can end up being reduced by LMR. Hence it may take more ply to find the best move. You mention aggressive and that is a new consideration. A weak evaluation will work less well the more aggressive the LMR is. I'm done here because Sopal's argument was time versus depth and you do not even address the argument.

https://www.chessprogramming.org/Michael_Sherwin
https://www.chessprogramming.org/Sherwin_Bitboards
https://www.chessprogramming.org/RomiChess

[OfekShochat]

Are you an engine author? Do you even know what LMR is?

https://www.chessprogramming.org/Tomasz_Sobczyk

Ouch, I wouldn't talk here for a month after that.
But just to summarize:
Depth is just a number. It doesn't say anything really. Especially between engines.
I will leave it at that. I don't want to add oil to the fire (is that a saying in English?). And tbh, people here can explain why better than I could ever do.