Pruning in PV nodes

[Sergei S. Markoff]

I find it hard to justify treating the first move differently from the rest of the moves...

I find it hard to argue against what all the top engines are doing, which is to treat PV and non-PV nodes differently.

That's not an argument. Fruit was a top engine, and applied LMR reductions solely based on the history counter values, not order or anything. Did that make it right? Does anybody do that today? (no).

Following that reasoning, nobody would be using LMR (none of the top engines are using any sort of reductions today and it is hard to argue against what they are doing...).

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

You're right, definitely it should be tested for each engine to determine — is it helps. For me disable reductions at PV node makes engine stronger (comparing to version with doing reductions at the same way as in non-pv nodes). Do you have another expirience with your engine?

[syzygy]

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

It's not the "crowd" that does it, it is the group of engines that are 100s of Elo stronger than the rest. This is not an argument based on lemmings but an argument based on objective measurement.

Maybe you do not understand why it works, but that does not change the fact that it works.

[bob]

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

It's not the "crowd" that does it, it is the group of engines that are 100s of Elo stronger than the rest. This is not an argument based on lemmings but an argument based on objective measurement.

Maybe you do not understand why it works, but that does not change the fact that it works.

And what is the proof that it DOES work? Fruit used history-based reductions. It was at the top of the rating list back then. Yet I took that history based reduction code and removed the history part completely. Didn't cost nor gain a single Elo. Just because the best program around splits the search into two parts, one for PV and one for non-PV does not mean it is correct. That's "lemming think". We've already gone through the null-move search stuff (don't do any null-moves along PV) and have seen that is wrong. Everyone has looked at the null-move idea to see why and how it works, and that is clearly understood. We've discussed things like fractional extensions for years. But to date no one has provided any evidence that extending by one amount on PV and by a different amount on non-PV nodes really works.

Hence my skepticism until I see or actually observe in my own testing, that this PV / non-PV being treated differently actually works and I understand why.

[bob]

I find it hard to justify treating the first move differently from the rest of the moves...

I find it hard to argue against what all the top engines are doing, which is to treat PV and non-PV nodes differently.

That's not an argument. Fruit was a top engine, and applied LMR reductions solely based on the history counter values, not order or anything. Did that make it right? Does anybody do that today? (no).

Following that reasoning, nobody would be using LMR (none of the top engines are using any sort of reductions today and it is hard to argue against what they are doing...).

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

You're right, definitely it should be tested for each engine to determine — is it helps. For me disable reductions at PV node makes engine stronger (comparing to version with doing reductions at the same way as in non-pv nodes). Do you have another expirience with your engine?

So far I have not run any test on any search enhancement where it performs better by doing it differently for PV and non-PV nodes. May be perfectly reasonable to split things that way, but I have not been able to find any benefit. If you have a specific thing to try, feel free and if it is not horribly complex, I'll make the changes and run a few million games to test it carefully...

[gladius]

I find it hard to justify treating the first move differently from the rest of the moves...

I find it hard to argue against what all the top engines are doing, which is to treat PV and non-PV nodes differently.

That's not an argument. Fruit was a top engine, and applied LMR reductions solely based on the history counter values, not order or anything. Did that make it right? Does anybody do that today? (no).

Following that reasoning, nobody would be using LMR (none of the top engines are using any sort of reductions today and it is hard to argue against what they are doing...).

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

You're right, definitely it should be tested for each engine to determine — is it helps. For me disable reductions at PV node makes engine stronger (comparing to version with doing reductions at the same way as in non-pv nodes). Do you have another expirience with your engine?

So far I have not run any test on any search enhancement where it performs better by doing it differently for PV and non-PV nodes. May be perfectly reasonable to split things that way, but I have not been able to find any benefit. If you have a specific thing to try, feel free and if it is not horribly complex, I'll make the changes and run a few million games to test it carefully...

Stockfish does quite a lot of things differently in PV/non-PV nodes, and removing that distinction has been tested on the framework for quite a few of them.

Things that have been tested and found to help:
1. No null move in PV nodes
2. Less aggressive LMR in PV nodes
3. Different IID parameters

Things I don't remember if have been tested:
1. No futility in PV
2. No razoring in PV
3. No prob-cut in PV

[syzygy]

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

It's not the "crowd" that does it, it is the group of engines that are 100s of Elo stronger than the rest. This is not an argument based on lemmings but an argument based on objective measurement.

Maybe you do not understand why it works, but that does not change the fact that it works.

And what is the proof that it DOES work?

How about having a look at any rating list and let reality sink in?

[bob]

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

It's not the "crowd" that does it, it is the group of engines that are 100s of Elo stronger than the rest. This is not an argument based on lemmings but an argument based on objective measurement.

Maybe you do not understand why it works, but that does not change the fact that it works.

And what is the proof that it DOES work?

How about having a look at any rating list and let reality sink in?

How about thinking for a bit and then letting THAT sink in. At one point all the top programs were using Fruit's history stuff. No longer. If we thought like you seem to think, we'd STILL be doing that. A chess program is the sum of a BUNCH of different parts. Nothing says that a few of those parts are not sub-optimal, it is the sum that is getting measured.

[syzygy]

Just because the "crowd" does it, doesn't make it right. MIGHT be, but NOT for THAT reason... If I run across a technical argument that is convincing, I might think differently. But I am definitely not a "lemming".

It's not the "crowd" that does it, it is the group of engines that are 100s of Elo stronger than the rest. This is not an argument based on lemmings but an argument based on objective measurement.

Maybe you do not understand why it works, but that does not change the fact that it works.

And what is the proof that it DOES work?

How about having a look at any rating list and let reality sink in?

How about thinking for a bit and then letting THAT sink in. At one point all the top programs were using Fruit's history stuff. No longer. If we thought like you seem to think, we'd STILL be doing that. A chess program is the sum of a BUNCH of different parts. Nothing says that a few of those parts are not sub-optimal, it is the sum that is getting measured.

So you actually believe this wasn't tested by the developers of Stockfish, Houdini, Komodo (despite numerous reports over the last years to the contrary, including some in this thread)?

[lucasart]

Having tested a lot of the PV node stuff in Stockfish, I can say that I 100% agree with Bob here!

A LOT of PV node dependant things have been removed, after careful testing. I can't even begin to count them.

The only PV node dependant stuff that is PROVEN to work in Stockfish is:
1/ skip Null search at PV nodes. That one is quite logical, because we should never fail high at a PV node, except for those artificial ones due to aspiration windows. The elo gain of this optimization is quite small, but measurable.
2/ IID reduction formula depends on PV-ness. PV nodes are reduces less than non PV nodes. The elo gain of this optimization is quite small, but measurable.

Even the LMR reduction depending on PV-ness is UNPROVEN. We cannot remove it because we need to retune perfectly the formula after that, and even then we would get an uncommitable 0 elo patch.

There are still lots of PV conditions, and the reason they cannot be removed is that our testing methodology does not allow it. The only way to commit a patch is if it is a clear elo gain, and unless it gains 2-3 elo, it's likely to fail in testing. Because PV nodes are few, and fewer as depth increases, tests removing PV node crap typically pass the SPRT with elo0=-1.5 elo1=4.5 at short TC, and fail the SPRT with elo0=0 elo1=6, because the elo gain is too small and decreases with depth (as PV nodes become scarce). This has misled Marco (and many others) to believe that these PV node conditions are inherently scalable, and now they are "sacred cows" of Stockfish, along with verification search, most of the endgame code, the broken unstoppable passer logic, etc.

Advice to anyone writing an engine: it's not because Stockfish does it that it's right. Testing is the only metric. Besides, some things work in some engines and not in others. Example: singular extension does not work for me, reducing cut nodes more than other nodes does not work either, etc. But these things really do work in SF.

[syzygy]

Having tested a lot of the PV node stuff in Stockfish, I can say that I 100% agree with Bob here!

100%, are you sure? Then how to explain that several PV-specific things have been PROVEN to work, as you just confirmed?

So which ones do not lose Elo if you enable them in the PV:
- razoring
- futility pruning
- null move
- probcut
- pruning at shallow depth
- LMR
- qsearch futility pruning
- qsearch pruning of negative SEE moves