Reductions from internal iterative deepening

[Evert]

Preliminary tests still inconclusive after ~5700 games.
At least it doesn't obviously hurt...

[Rebel]

You assume a different IID implementation, one that actually has >1 iterations. That is of course the original meaning of the name "IID" but today it seems that many programmers, including Evert, just do one Search() call with reduced depth and no iterations there.

Precisely. IID comes in many forms. I use a one step approach of RD/2 (with a minimum RD of 4) and then research at full depth. Various tries to reduce or extend the research failed. One tries the craziest ideas, you never know for sure if you haven't tried.

[syzygy]

Hgm's point is that your proposed reduction (of all moves of the current node) has more or less the same effect as a reduction performed in the parent for the move leading to the current position. That turns your fail low condition for the current node into a fail high condition for the parent node. Your proposal is to not search further (i.e. not do the research) if IID returned a value below alpha.

Wrong.
If IID indicates that the node will likely fail low, then the search is reduced. That's the idea but the devil is, as always, in the details.

So what is wrong? If your details make it wrong, maybe you should have added them in your initial post?

That parent node is not going to do the research either, because it had no idea the move was being reduced through your IID reduction. (Or if the parent had reduced the move, your IID reduction is reducing it once more.)

Sure, but I don't see how that is unique to this proposed scheme: any move that is reduced might not have failed low if searched at full depth. That's why reducing arbitrarily is a bad idea.

The point is that your scheme, viewed from the parent node, is similar to an LMR reduction, except that it foregoes on the research upon a fail high which I believe most think is necessary.

Btw, why only reduce when you use IID and not also if you find the node in the TT but with insufficient depth (but with a score that would otherwise cause a cut-off)?

[Evert]

So what is wrong?

This

Your proposal is to not search further (i.e. not do the research) if IID returned a value below alpha.

is not what I suggested.

If your details make it wrong, maybe you should have added them in your initial post?

Your snarkyness is misplaced, particularly since I already elaborated on some of the search peculiarities in Jazz. Besides, this is an idea, a suggestion. I didn't say anywhere that this is the perfect and exact final version of it.

The point is that your scheme, viewed from the parent node, is similar to an LMR reduction, except that it foregoes on the research upon a fail high which I believe most think is necessary.

I already explained why this is actually not what it is.
However, the more I think about it, the more it seems to me that the idea is really not very different from LMR (at the current node that is) and so it may well be redundant with that. I haven't tried tweaking LMR based on the results from IID but that could be worth looking into more.

Btw, why only reduce when you use IID and not also if you find the node in the TT but with insufficient depth (but with a score that would otherwise cause a cut-off)?

No reason; in fact I mentioned this inconsistency in an earlier post.
The main difference would be with respect to the search-bounds: when I do IID, I use the current [alpha, beta], which will in general be different from what was used to generate the TT entry.

[hgm]

However, the more I think about it, the more it seems to me that the idea is really not very different from LMR (at the current node that is) and so it may well be redundant with that.

This is still something I don't get. The essence of LMR is that you only reduce 'late' (= presumably useless) moves. The way you described it sounded like you would reduce all moves, if an earlier iteration would suggest you are in an all-node. And that would make it very different. Reducing all moves in a node often boils down to just another way to define depth.

[Evert]

This is still something I don't get. The essence of LMR is that you only reduce 'late' (= presumably useless) moves. The way you described it sounded like you would reduce all moves, if an earlier iteration would suggest you are in an all-node.

The (implicit) assumption you make with LMR is that you are in an all-node once you get to the "late" move stage. If you have reason to believe that the current node is not an all-node (despite having gone far down the move list), then you should not apply LMR because you expect one of the moves to improve alpha and reducing it may hide that.

This is where the similarity comes in: you reduce the search when you have reason to believe that the current node is an all-node. The difference is in when you think you have a reason to believe that: either while going through the move list, or before you start.

[hgm]

Well, that is not my idea of LMR. I would like to see branches with reasonable moves to be searched at full depth, and only nonsense to be reduced. Because in a deep tree there are many branches that end in scores just marginally below that of the PV, and upon elongation might actually greatly exceed it, and become the new PV. And, not being on the previous PV, they would go through a lot of all nodes.

By reducing everything, you won't be able to see superior alternative lines before you also reach the same depth as where the superiority shows for all of the garbage branches. And that makes it incredibly expensive to find anything. What you need is a method that reduces the garbage, but has not too much impact on reasonable alternatives.

[Evert]

Well, that is not my idea of LMR. I would like to see branches with reasonable moves to be searched at full depth, and only nonsense to be reduced.

Sure, but doesn't that just become semantics?

Because in a deep tree there are many branches that end in scores just marginally below that of the PV, and upon elongation might actually greatly exceed it, and become the new PV. And, not being on the previous PV, they would go through a lot of all nodes.

Evidently, but that is true of any reduction scheme.

By reducing everything, you won't be able to see superior alternative lines before you also reach the same depth as where the superiority shows for all of the garbage branches. And that makes it incredibly expensive to find anything. What you need is a method that reduces the garbage, but has not too much impact on reasonable alternatives.

Again, that's true of any reduction scheme, so I fail to see how it would be a specific problem with "IID reductions"?

Note that the main idea is not "reduce everything" (as I said already, the first move is excluded anyway for instance) but the condition used for making reduction decisions.

[hgm]

Let me make sure I understand this, then:

If your IID pilot search (at depth = d-4, say) fails low, you would do a d-1 iteration for all moves except the first (which you would search at d), and then call it quits? (If they all failed low)?

That would indeed make it look very much like LMR, except that you would not do the LMR when the pilot search did not fail low.

I am not sure how such a conditional application of LMR would be helpful. If a move in the pilot search fails high it causes a beta cutoff there, and you would immediately proceed with that move at full depth, as you would sort it in front. If it fails high again you are done. But if it fails low now, I see no reason why you would expect any of the other moves to fail high in its place. or have more chance to switch from fail low at d-1 to fail high at d than when the first move would not have dropped in score to below alpha. So I would prefer to reduce as always.