forbidden sequel

[hgm]

I am still looking for a cheap trick to solve the following problem:

[d]8/1p6/n1p5/8/8/2P5/1P6/R7 w

Suppose I have a position that contains a pattern like above. On the King side I am planning an attack, and at 15 ply this looks pretty good, with a score of, say, +3. Unfortunately, when I search at 16 ply, that plan goes bust, and the old PV move now scores -1. 'Fortunately' it finds a better move, Rxa6, followed by Pxa6, and then exactly the same sequence of moves as what gave me +3 at 15 ply. Since I prefixed it with an unrelated exchange sac, this now of course scores only +1. But that is still a lot better than -1. Would you believe it?

Of course what likely happens is that at 17 ply Rxa6 still scores +1, but at 18 ply, the sequel goes bust just as it did when I played it immediately, and with the pointless exchange sac added, it now drops from +1 to -3. If I was lucky, I reached 18 ply before it was time to move. Then I just lose an enormous amount of search effort on switching PV twice. If I have to move at 16 or 17 ply, I play a blunder.

Does this happen often enough to worry about? Well, perhaps not in orthodox Chess. But I am currently dealing with a variant where the pattern basically is the initial position, where you can sac a valuable piece for a less valuable, undeveloped and totally inactive one, which is almost guaranteed not to have any side effects other than just burning material to eat away 2 ply. So it happens all the time the score makes a big swing, because some interesting tactic is coming within the horizon. It forces me to analyze 2 ply deeper than when the swing is first seen, to make sure the sac actually isn't a brilliant tempo-gaining move that solves the real problem. (Which it sometimes is, e.g. because the piece I sac was blocking some other piece I needed.) That can take very long.

But if I can expect a 'switch to nonsense', it should be possible for the engine to se it too. The pattern is pretty conspicuous: drop in PV score, followed by a switch to an unfavorable exchange prefixed to that same PV (or actually the PV of the previous ply), with a score that just deducts the sacrificed material from that. It seems better to ignore that score for the next two iterations.

This is a bit tricky to implement, though. For one, if I would just do it in the root, it would not be helpful. Because it can interject the delaying sac at any later point in the PV as well. So even in a node somewhat further along the PV these delaying tactics should be ignored. But deepening in such nodes occurs through revisiting them in the next root iteration, the only info left from that iteration being the score and the first move of the PV. This is a bit meagre. And the main problem is that we not only need to know whether the PV score drops this iteration, but also whether it dropped on the iteration before.

So it seems we would need to store extra info in the TT for PV nodes. Now this is advisable anyway. It doesn't take much space, as PV nodes are relatively rare. So the extra info can be put in a small hash table only used for PV nodes. Crafty, for instance, hashes the entire PV that way, rather than just a hash move. With not too much effort we could store the PVs and scores of the last two iterations there. Then we could directly detect a score drop compared to two iterations ago, and for any replacement move detect if it is planning to follow it up with the same PV as two iterations ago, with the expected score. This would be the suspect situation, where we should either ignore the better score, or extend the search of that move 2 ply (which probably is too expensive).

We could make the procedure more selective, by explicitly testing whether there is a 'tactical connection' between the first two (sacrificial) ply, and the PV after the score drop (i.e. searched in this node, or stored from the previous iteration in the TT, depending on where the score drop took place). If there is, we can assume that the sacrifice does affect the refutation that caused the score drop, and thus needs extension, while if it doesn't, we could simply ignore the score.

[cdani]

Does this happen often enough to worry about? Well, perhaps not in orthodox Chess

Is possible that in orthodox chess this happens often, but as the current engines are so strong, let's say the typical combination where this is important lasts most times up to 10 moves, and any strong engine sees much farther than this.

If this is like this, means that an engine needs not to deal with this problem once is strong enough to statistically overcome most of the times that this happen.

This is the same that happened many years ago, when the engines where much weaker and they implemented tricks to detect basic tactical stuff, tricks that then just disappeared from the code as they achieved higher depths that make them unnecessary.

I don't know if this is possible to happen in the variant you are talking about. So maybe just going for raw increased strength will be statistically enough to solve this stuff.

[AlvaroBegue]

I was going to say something similar to Daniel José. I have a couple more things to say.

[...] where you can sac a valuable piece for a less valuable, undeveloped and totally inactive one, which is almost guaranteed not to have any side effects other than just burning material to eat away 2 ply.

That part sounds like something the evaluation function should deal with. Perhaps having an evaluation function too close to simplistic bean counting is making the situation worse.

One last thing: Perhaps you could try spending a little more time if the current best move is an inferior exchange. That way you'll have a better chance to see the trouble ahead.