LMR and 'tactically connected' moves

[hgm]

I still have a problem in understanding how to apply the criterion of "tacticaly connected" moves to decide which late moves to exempt from reduction even if they fail low, and which not. What I do understand is this:

When I do a late move, and the opponent's null-move reply fails low because of the piece I moved now captures something, I should undo the reduction. Apparently the move did threaten something important enough to swing the score in my advantage, which leads to a more or less forced reply of the opponent to prevent that, and thus enhances the possibility that I get the time to do something dangerous deeper in the tree that he had no opportunity to pre-empt. So searching deep is not a wasted effort then.

The conceptual problem I have is this: when I threaten something that could swing the score, e.g. I attack a Bishop through a Pawn push, and the opponents ignores it and null moves... Then it is not at all certain that the null-move refutation will be PxB. If a Q or R trade is possible in the position, the best move ordering from QS tree-size perspective is to first disarm the opponent by making the QxQ or RxR capture, and harvest the Bishop after his recapture. So the theoretically correct refutation (and perhaps the only one if my own Q or R were not defended, so the PxB will be met with his QxQ or RxR) will be my QxQ or RxR. This will not be recognized as tactically connected to the Pawn push / attack on the Bishop, unless I look deeper in the null-move tree. So how would I decide that I should not reduce the Pawn push?

One reasoning I could think of is this; I push a Pawn, and after that, QxQ is devastating for the oppoent if he doesn't act. If the QxQ seems tactically not connectd to the Pawn push, I must have been able to play it before, in stead of that Pawn push. (Otherwise the tactical connection would be pretty obvious.) This means I must have searched it before in the current node, as (good or equal) captures are searched before Pawn pushes. (If the move in question would not have been a Pawn puh, but a capture itself, it would not be a candidate for reduction anyway, and the problem would not arise.) Apparently it did not fail high then, or we would not be searching the Pawn push now. But after the Pawn push, the QxQ does fail high. Isn't this fact in itself not sufficient proof that the QxQ must be tactically connected to the Pawn push (even if we can't easily see how)? What more proof could we want?

But if we accept this reasoning, the LMR pretty much boils down to an extra large null-move reduction. E.g. R=3, except after hash move or killer, where we do R=2. And of course after a capture, but after a capture we are unlikely to get a fail high if we do not recapture, i.e. the null move would be doomed in any case, and it is immateial how much you reduce it. Unless the capture was in fact itself a recapture. And then we might cast doubt on the wisdom of exempting t from reduction, as the original idea of not reducing captures what that they tie the hands of the opponent, while in fact the need to recapture was keeping ourselves busy and unable to pre-empt future malice (a fact already accounted for by not reducing the capture preceeding it, forcing us to recapture rather than null moving).

[bob]

I assume you were following a past discussion from Tord? In any case, I've experimented with this (along with a ton of other things) and I have yet to find anything that helps. If you do some test positions you can be quickly misled. I was sure a couple of the recent ideas I tried would be good because they helped on tactical test positions. One change might add 2 more correct out of 100 hard positions, or find an easier solution in 1/4 the time. But put it in real games and it hurt every last time...

[diep]

I still have a problem in understanding how to apply the criterion of "tacticaly connected" moves to decide which late moves to exempt from reduction even if they fail low, and which not. What I do understand is this:

When I do a late move, and the opponent's null-move reply fails low because of the piece I moved now captures something, I should undo the reduction. Apparently the move did threaten something important enough to swing the score in my advantage, which leads to a more or less forced reply of the opponent to prevent that, and thus enhances the possibility that I get the time to do something dangerous deeper in the tree that he had no opportunity to pre-empt. So searching deep is not a wasted effort then.

The conceptual problem I have is this: when I threaten something that could swing the score, e.g. I attack a Bishop through a Pawn push, and the opponents ignores it and null moves... Then it is not at all certain that the null-move refutation will be PxB. If a Q or R trade is possible in the position, the best move ordering from QS tree-size perspective is to first disarm the opponent by making the QxQ or RxR capture, and harvest the Bishop after his recapture. So the theoretically correct refutation (and perhaps the only one if my own Q or R were not defended, so the PxB will be met with his QxQ or RxR) will be my QxQ or RxR. This will not be recognized as tactically connected to the Pawn push / attack on the Bishop, unless I look deeper in the null-move tree. So how would I decide that I should not reduce the Pawn push?

One reasoning I could think of is this; I push a Pawn, and after that, QxQ is devastating for the oppoent if he doesn't act. If the QxQ seems tactically not connectd to the Pawn push, I must have been able to play it before, in stead of that Pawn push. (Otherwise the tactical connection would be pretty obvious.) This means I must have searched it before in the current node, as (good or equal) captures are searched before Pawn pushes. (If the move in question would not have been a Pawn puh, but a capture itself, it would not be a candidate for reduction anyway, and the problem would not arise.) Apparently it did not fail high then, or we would not be searching the Pawn push now. But after the Pawn push, the QxQ does fail high. Isn't this fact in itself not sufficient proof that the QxQ must be tactically connected to the Pawn push (even if we can't easily see how)? What more proof could we want?

But if we accept this reasoning, the LMR pretty much boils down to an extra large null-move reduction. E.g. R=3, except after hash move or killer, where we do R=2. And of course after a capture, but after a capture we are unlikely to get a fail high if we do not recapture, i.e. the null move would be doomed in any case, and it is immateial how much you reduce it. Unless the capture was in fact itself a recapture. And then we might cast doubt on the wisdom of exempting t from reduction, as the original idea of not reducing captures what that they tie the hands of the opponent, while in fact the need to recapture was keeping ourselves busy and unable to pre-empt future malice (a fact already accounted for by not reducing the capture preceeding it, forcing us to recapture rather than null moving).

Hi Harm-Geert,

Your observation is very correct that LMR in most cases increases nullmove R=3 to R=4 and last few plies nullmove R=2 to R=3.

So when i claim that LMR doesn't really win big search depth for Diep, that's because when i do LMR in diep i'm using nullmove R=3/2 and when not using LMR i run with nullmove R=3.

At depthleft >= 6, increasing nullmove from R=3 to R=4 in itself doesn't really decrease the total search tree bigtime. Maybe 1% or so. Real tiny.

This is why my idea now is to investigate once again pruning somehow faster last few plies with Diep instead of LMR. Of course in combination with nullmove R=3.

The last few plies IMHO being a tad more selective is a possibility. You run a few plies risk, whereas LMR has a whole range of risks in it.

My original intention back in 2005 was to do the search replacement last 5 ply.

That really searches a lot of plies deeper.

Let's be very fair here. If we're up 3 pawns or so and have 5 ply depthleft to go. Why not prune if we have some sort of safety that the opponent has no threats?

Vincent

[bob]

I still have a problem in understanding how to apply the criterion of "tacticaly connected" moves to decide which late moves to exempt from reduction even if they fail low, and which not. What I do understand is this:

When I do a late move, and the opponent's null-move reply fails low because of the piece I moved now captures something, I should undo the reduction. Apparently the move did threaten something important enough to swing the score in my advantage, which leads to a more or less forced reply of the opponent to prevent that, and thus enhances the possibility that I get the time to do something dangerous deeper in the tree that he had no opportunity to pre-empt. So searching deep is not a wasted effort then.

The conceptual problem I have is this: when I threaten something that could swing the score, e.g. I attack a Bishop through a Pawn push, and the opponents ignores it and null moves... Then it is not at all certain that the null-move refutation will be PxB. If a Q or R trade is possible in the position, the best move ordering from QS tree-size perspective is to first disarm the opponent by making the QxQ or RxR capture, and harvest the Bishop after his recapture. So the theoretically correct refutation (and perhaps the only one if my own Q or R were not defended, so the PxB will be met with his QxQ or RxR) will be my QxQ or RxR. This will not be recognized as tactically connected to the Pawn push / attack on the Bishop, unless I look deeper in the null-move tree. So how would I decide that I should not reduce the Pawn push?

One reasoning I could think of is this; I push a Pawn, and after that, QxQ is devastating for the oppoent if he doesn't act. If the QxQ seems tactically not connectd to the Pawn push, I must have been able to play it before, in stead of that Pawn push. (Otherwise the tactical connection would be pretty obvious.) This means I must have searched it before in the current node, as (good or equal) captures are searched before Pawn pushes. (If the move in question would not have been a Pawn puh, but a capture itself, it would not be a candidate for reduction anyway, and the problem would not arise.) Apparently it did not fail high then, or we would not be searching the Pawn push now. But after the Pawn push, the QxQ does fail high. Isn't this fact in itself not sufficient proof that the QxQ must be tactically connected to the Pawn push (even if we can't easily see how)? What more proof could we want?

But if we accept this reasoning, the LMR pretty much boils down to an extra large null-move reduction. E.g. R=3, except after hash move or killer, where we do R=2. And of course after a capture, but after a capture we are unlikely to get a fail high if we do not recapture, i.e. the null move would be doomed in any case, and it is immateial how much you reduce it. Unless the capture was in fact itself a recapture. And then we might cast doubt on the wisdom of exempting t from reduction, as the original idea of not reducing captures what that they tie the hands of the opponent, while in fact the need to recapture was keeping ourselves busy and unable to pre-empt future malice (a fact already accounted for by not reducing the capture preceeding it, forcing us to recapture rather than null moving).

Hi Harm-Geert,

Your observation is very correct that LMR in most cases increases nullmove R=3 to R=4 and last few plies nullmove R=2 to R=3.

So when i claim that LMR doesn't really win big search depth for Diep, that's because when i do LMR in diep i'm using nullmove R=3/2 and when not using LMR i run with nullmove R=3.

At depthleft >= 6, increasing nullmove from R=3 to R=4 in itself doesn't really decrease the total search tree bigtime. Maybe 1% or so. Real tiny.

This is why my idea now is to investigate once again pruning somehow faster last few plies with Diep instead of LMR. Of course in combination with nullmove R=3.

The last few plies IMHO being a tad more selective is a possibility. You run a few plies risk, whereas LMR has a whole range of risks in it.

My original intention back in 2005 was to do the search replacement last 5 ply.

That really searches a lot of plies deeper.

Let's be very fair here. If we're up 3 pawns or so and have 5 ply depthleft to go. Why not prune if we have some sort of safety that the opponent has no threats?

Vincent

Since I have added checks to q-search, I always use R=3 for null-move. I tested this carefully to ensure it was better than R=3~2 that I previously used. LMR still adds another +40 elo on top of that however, at least for me.

[diep]

I still have a problem in understanding how to apply the criterion of "tacticaly connected" moves to decide which late moves to exempt from reduction even if they fail low, and which not. What I do understand is this:

When I do a late move, and the opponent's null-move reply fails low because of the piece I moved now captures something, I should undo the reduction. Apparently the move did threaten something important enough to swing the score in my advantage, which leads to a more or less forced reply of the opponent to prevent that, and thus enhances the possibility that I get the time to do something dangerous deeper in the tree that he had no opportunity to pre-empt. So searching deep is not a wasted effort then.

The conceptual problem I have is this: when I threaten something that could swing the score, e.g. I attack a Bishop through a Pawn push, and the opponents ignores it and null moves... Then it is not at all certain that the null-move refutation will be PxB. If a Q or R trade is possible in the position, the best move ordering from QS tree-size perspective is to first disarm the opponent by making the QxQ or RxR capture, and harvest the Bishop after his recapture. So the theoretically correct refutation (and perhaps the only one if my own Q or R were not defended, so the PxB will be met with his QxQ or RxR) will be my QxQ or RxR. This will not be recognized as tactically connected to the Pawn push / attack on the Bishop, unless I look deeper in the null-move tree. So how would I decide that I should not reduce the Pawn push?

One reasoning I could think of is this; I push a Pawn, and after that, QxQ is devastating for the oppoent if he doesn't act. If the QxQ seems tactically not connectd to the Pawn push, I must have been able to play it before, in stead of that Pawn push. (Otherwise the tactical connection would be pretty obvious.) This means I must have searched it before in the current node, as (good or equal) captures are searched before Pawn pushes. (If the move in question would not have been a Pawn puh, but a capture itself, it would not be a candidate for reduction anyway, and the problem would not arise.) Apparently it did not fail high then, or we would not be searching the Pawn push now. But after the Pawn push, the QxQ does fail high. Isn't this fact in itself not sufficient proof that the QxQ must be tactically connected to the Pawn push (even if we can't easily see how)? What more proof could we want?

But if we accept this reasoning, the LMR pretty much boils down to an extra large null-move reduction. E.g. R=3, except after hash move or killer, where we do R=2. And of course after a capture, but after a capture we are unlikely to get a fail high if we do not recapture, i.e. the null move would be doomed in any case, and it is immateial how much you reduce it. Unless the capture was in fact itself a recapture. And then we might cast doubt on the wisdom of exempting t from reduction, as the original idea of not reducing captures what that they tie the hands of the opponent, while in fact the need to recapture was keeping ourselves busy and unable to pre-empt future malice (a fact already accounted for by not reducing the capture preceeding it, forcing us to recapture rather than null moving).

Hi Harm-Geert,

Your observation is very correct that LMR in most cases increases nullmove R=3 to R=4 and last few plies nullmove R=2 to R=3.

So when i claim that LMR doesn't really win big search depth for Diep, that's because when i do LMR in diep i'm using nullmove R=3/2 and when not using LMR i run with nullmove R=3.

At depthleft >= 6, increasing nullmove from R=3 to R=4 in itself doesn't really decrease the total search tree bigtime. Maybe 1% or so. Real tiny.

This is why my idea now is to investigate once again pruning somehow faster last few plies with Diep instead of LMR. Of course in combination with nullmove R=3.

The last few plies IMHO being a tad more selective is a possibility. You run a few plies risk, whereas LMR has a whole range of risks in it.

My original intention back in 2005 was to do the search replacement last 5 ply.

That really searches a lot of plies deeper.

Let's be very fair here. If we're up 3 pawns or so and have 5 ply depthleft to go. Why not prune if we have some sort of safety that the opponent has no threats?

Vincent

Since I have added checks to q-search, I always use R=3 for null-move. I tested this carefully to ensure it was better than R=3~2 that I previously used. LMR still adds another +40 elo on top of that however, at least for me.

Checks indeed improve things dramatically, congrats.

For Diep doing R=3 last few plies is way too dangerous when combined with LMR. It starts to miss too much.

I still might do an experiment playing games with R=2 in combination with LMR versus R=3 without LMR.

Yet obviously R=3 at plydepth 4 is a lot faster than R=2 there.
At depths 5 and above savings are a lot smaller from R=2 to R=3.

It's just these last few plies that i smell already for 15 years we can do things more selective

Now i recalculate, yes it really has been more than 15 years already that i'm into computerchess programming and i didn't get grey hair from it yet

Vincent