LMR and null move selectivity

[Don]

There has been some discussion on this already, but it's scattered all over the place.

Has anyone had experience with forbidding null move when inside an LMR reduced search (at any recursion level) or forbidding LMR when inside a null move search - or BOTH?

I will probably test all 4 combinations to see what happens, but I'm interested in the thoughts and experiences of others who may already have looked at this.

[mjlef]

There has been some discussion on this already, but it's scattered all over the place.

Has anyone had experience with forbidding null move when inside an LMR reduced search (at any recursion level) or forbidding LMR when inside a null move search - or BOTH?

I will probably test all 4 combinations to see what happens, but I'm interested in the thoughts and experiences of others who may already have looked at this.

I have experimented with this. But never do enough testing to be sure of much, well, until recently.

Some ideas:

a. Hold off the depth reduction until entering the NULL search code. If the LMR conditions are met, use a depth reduction of R+1 for the null search instead of just R. Testing showed this was close, but seemed a bit worse, possibly because in the previous node, something bad was afoot and no move would be good enough to get above Alpha. But if you use the current eval being above Beta before trying NULL, it gets run in fewer cases and so you never do an LMR at that node.

b. Ban NULL after a null move search

c. Ban LMR at the node just under a null search (the idea being you already reduced depth by like 4 ply, so making it 5 might be too risky).

I would love to see what you test and find out.

Mark

[bob]

There has been some discussion on this already, but it's scattered all over the place.

Has anyone had experience with forbidding null move when inside an LMR reduced search (at any recursion level) or forbidding LMR when inside a null move search - or BOTH?

I will probably test all 4 combinations to see what happens, but I'm interested in the thoughts and experiences of others who may already have looked at this.

Yep. So far any attempt to reduce reductions has also reduced Elo. I have one attempt that is showing a small amount of promise, but I'm not going to say more until testing proves it is better. This is actually a case Tracy and I talked about months ago where I had suggested that reducing moves that appear to be really ugly might become more aggressive. I am still playing with the definition of "ugly" (based on Evaluate() and / or Swap()). So far I have coaxed maybe another 10 Elo out of this, so nothing remarkable yet. But I have more things to try/test before considering it "done". Soon as it is confirmed to be better, I'll post the results, but I don't want to send anyone off on a wild-goose-chase until I am reasonably sure of the results... One person chasing wild geese is more than enough.

[diep]

There has been some discussion on this already, but it's scattered all over the place.

Has anyone had experience with forbidding null move when inside an LMR reduced search (at any recursion level) or forbidding LMR when inside a null move search - or BOTH?

I will probably test all 4 combinations to see what happens, but I'm interested in the thoughts and experiences of others who may already have looked at this.

I tried even more than just this, i also tried to reduce the R of a nullmove right after a reduction, or undo the reduction when doing the nullmove (that's a different thing, think about it!).

My conclusion was that all such rules are total hopeless, the only basic mistake LMR makes which is real ugly, is that the odds the best moves are inside the move selection that's not getting reduced is rather tiny, when it is not some trivial tactical move.

Vincent

[mjlef]

Bob and Don,

Did you ever finish your tests? What did you find?

Mark

[bob]

There has been some discussion on this already, but it's scattered all over the place.

Has anyone had experience with forbidding null move when inside an LMR reduced search (at any recursion level) or forbidding LMR when inside a null move search - or BOTH?

I will probably test all 4 combinations to see what happens, but I'm interested in the thoughts and experiences of others who may already have looked at this.

I beat that to death months ago. I found _no_ case where restricting null-move or LMR improved the scores. I found a _ton_ of cases where restricting either or both hurt.

[BubbaTough]

This is a good idea to try...I don't remember trying it before. My instinct is that engines with conservative null move / LMR depths will not benefit from being even more conservative...but that maybe folks with more aggressive approaches (say, folks that ramp up LMR and null move a lot depending on position) might. Thus, I expect it would hurt engines like Crafty or DIEP, but perhaps engines like LearningLemming, Weid, or ZCT (or Now?) would benefit. I will run a few tests and report back (nothing as comprehensive as Bob's test of course).

-Sam

EDIT:

Preliminary result 1. Tactical testbed of 86 moves at 8 seconds a move:

Normal: 69 / 86 4:34
don't LMR when null moving: 66/86 4:59

Hmmm....not that promising. I'll try some games

[Mincho Georgiev]

I beat that to death months ago. I found _no_ case where restricting null-move or LMR improved the scores. I found a _ton_ of cases where restricting either or both hurt.

I definitely agree. I did A LOT of test some time ago and stopping the null move ability during LMR at least in my program appears to be a very bad idea. Maybe more interesting are the tests between LMR at pv nodes and LMR omitted at pv nodes, because that's the point when i get controversial results.

[bob]

I beat that to death months ago. I found _no_ case where restricting null-move or LMR improved the scores. I found a _ton_ of cases where restricting either or both hurt.

I definitely agree. I did A LOT of test some time ago and stopping the null move ability during LMR at least in my program appears to be a very bad idea. Maybe more interesting are the tests between LMR at pv nodes and LMR omitted at pv nodes, because that's the point when i get controversial results.

I have tried limiting LMR and/or NULL searches at PV nodes. I found absolutely no way to improve results by doing so. In fact, every limiting approach I tried hurt overall performance, which was not exactly what I expected when I first tried this maybe 10-12 years ago (no null on a PV node). I spent 2 months earlier this year/late last year working with ways to limit null and/or LMR. I tried the current history approaches, and even reported that when I removed the current history counters from existing programs (such as Toga) it made absolutely no difference with respect to their strength when tested on my cluster. I gave up on any sort of history counter approach a year or two ago, but went back and retested carefully back earlier this year. And found nothing that helped. I then decided to modify existing programs that use history counter stuff to make reduction decisions (glaurung, toga, fruit, etc) and found the same result there. It doesn't work. Doesn't seem to hurt. But doesn't help either.

[Mincho Georgiev]

Doesn't seem to hurt. But doesn't help either.

That's true when it comes down to LMR. For null move i would say that there is a noticeable difference.I mean "if(!pvnode)" for null move definitely hurts. I'm talking about my own tests of course. May be someone would disagree.