Singular Extensions revisited

[yoshiharu]

I could not tell a difference, but this was prior to cluster testing days. I saved the "singular" test source code to try again one day. ...

I will wait

There is also another version that has some potential, the old null-move threat detection idea. Goes like this:

When a move fails high, before you return beta, you do a quick null-move search (normal null-move but depth reduction amount is open for debate). If the null-move search fails low, you now have a quandary. The current move fails high. Doing nothing fails low.

I'm probably missing something, but aren't you supposed to have this information already? If the null move you already searched fails low, and find a fail high nonetheless, isn't that enough to trigger this threat detection?

Cheers, mr

[bob]

I could not tell a difference, but this was prior to cluster testing days. I saved the "singular" test source code to try again one day. ...

I will wait

There is also another version that has some potential, the old null-move threat detection idea. Goes like this:

When a move fails high, before you return beta, you do a quick null-move search (normal null-move but depth reduction amount is open for debate). If the null-move search fails low, you now have a quandary. The current move fails high. Doing nothing fails low.

I'm probably missing something, but aren't you supposed to have this information already? If the null move you already searched fails low, and find a fail high nonetheless, isn't that enough to trigger this threat detection?

Cheers, mr

Not quite. Lots of null-move searches fail low because you use the same window as the normal search. However, if you shift the alpha/beta window down by (say) 1/2 pawn, now it really should fail high all the time. If it doesn't, then this position is not just simply "not great", it is really "not good at all". And _that_ is what triggers the extension. If you want to experiment, take a program that uses null-move and make the simple change so that when a null-move search fails low, before you continue the normal search and start trying real moves, do a second null-move search with an offset window like beta-pawn and beta-pawn+1. This will often fail high when the other one did not, because of the pawn margin. If it doesn't, then this position must be pretty bad. And this is what we would use to trigger the extension.

Again, the idea is that one move fails high, a downward-offset null-move search fails low, which is a hint that this one move is really holding off some threat of the opponent.

[yoshiharu]

Not quite. Lots of null-move searches fail low because you use the same window as the normal search. However, if you shift the alpha/beta window down by (say) 1/2 pawn, now it really should fail high all the time. If it doesn't, then this position is not just simply "not great", it is really "not good at all". And _that_ is what triggers the extension.

Ok, now I know what I missed
I implemented a trick like this when I wrote the null-move part of my engine, but I had so many issues to fix elsewhere, that I decided to keep it simple, for the moment.
So, do you suggest to do this threat detection only in case of a fail high (re-search at increased depth if triggered) or at every given node (avoids re-searches, but extends every move also in desperate positions)?
Intuitively I would prefer the former, but maybe there are issues with re-searches...

Cheers, mr

[bob]

Not quite. Lots of null-move searches fail low because you use the same window as the normal search. However, if you shift the alpha/beta window down by (say) 1/2 pawn, now it really should fail high all the time. If it doesn't, then this position is not just simply "not great", it is really "not good at all". And _that_ is what triggers the extension.

Ok, now I know what I missed
I implemented a trick like this when I wrote the null-move part of my engine, but I had so many issues to fix elsewhere, that I decided to keep it simple, for the moment.
So, do you suggest to do this threat detection only in case of a fail high (re-search at increased depth if triggered) or at every given node (avoids re-searches, but extends every move also in desperate positions)?
Intuitively I would prefer the former, but maybe there are issues with re-searches...

Cheers, mr

I only did it on fail-high (or PV of course) nodes, because there is no benefit to doing it on an ALL node since none will be candidates as a "best move" to trigger the extension. I deferred the downward-offset null-window search until I knew I had a "best move" that might need extension, to avoid doing the offset null-window search on ALL nodes where it is simply a waste of time.

The two null-window searches we do are different. At the top of search, you usually use beta-1, beta for the null-window search to get out of searching anything should this fail high. This search uses beta-pawn-1 and beta-pawn (the "pawn" value can be varied. The smaller the value, the more extensions you will trigger since it is more likely to fail low with higher alpha bounds. We are reducing the alpha bound already. As you lower the window, the chance of a fail low drops and the total number of these extensions drop. In looking at my old code, the margin I used was -1.5 pawns it turns out. Whether that was optimal or not I do not know. I have this on my list of things to check at some point in time, when I run out of other ideas.