Returning values of beta cutoffs

[Chan Rasjid]

A simple rule of thumb for fail-soft:
It can return a score x outside of bounds x < alpha or x > beta provided x is a score returned by a search done, x = -search(...).

All other "artificial" fail-low or fail high, eg futility prune, null-move, etc, should fail with the actual bound.

Rasjid.

[michiguel]

I wouldn't say that fail soft can be a source of bugs per se. But maybe it can expose latent bugs which are already in your code.

Returning score without taking into account the safety margin in futility pruning nodes or quies nodes with delta pruning is a bug per se.

Miguel

[rbarreira]

If you can't guarantee a score is correct you shouldn't return it. But that shouldn't be blamed on fail-soft, which in itself is a 100% correct version of alpha-beta.

[michiguel]

If you can't guarantee a score is correct you shouldn't return it. But that shouldn't be blamed on fail-soft, which in itself is a 100% correct version of alpha-beta.

Fail soft is 100% correct if you don't do pruning, or you don't use a hash table. Once you start pruning + HT, the basic assumptions of fail soft are not valid anymore, unless you properly adjust it.

Miguel

[rbarreira]

If you can't guarantee a score is correct you shouldn't return it. But that shouldn't be blamed on fail-soft, which in itself is a 100% correct version of alpha-beta.

Fail soft is 100% correct if you don't do pruning, or you don't use a hash table. Once you start pruning + HT, the basic assumptions of fail soft are not valid anymore, unless you properly adjust it.

Miguel

Once you start pruning the basic assumptions of alpha-beta aren't valid anymore either, this doesn't make alpha-beta a flawed algorithm. It makes the pruning "flawed", which means you have to take additional precautions, and even then you'll still see "bugs" from time to time.

[michiguel]

If you can't guarantee a score is correct you shouldn't return it. But that shouldn't be blamed on fail-soft, which in itself is a 100% correct version of alpha-beta.

Fail soft is 100% correct if you don't do pruning, or you don't use a hash table. Once you start pruning + HT, the basic assumptions of fail soft are not valid anymore, unless you properly adjust it.

Miguel

Once you start pruning the basic assumptions of alpha-beta aren't valid anymore either, this doesn't make alpha-beta a flawed algorithm. It makes the pruning "flawed", which means you have to take additional precautions, and even then you'll still see "bugs" from time to time.

You are trying find a blame in a specific technique but I don't. I am trying to warn the poster of the dangers of doing FS. If you fail soft returning score + pruning + HT, you have a bug. I do not blame FS, I blame the programmer

Miguel

[bob]

It might be a silly question, but there we go...
By inspecting chess programs sources, whenever occurs a beta cutoff some of them:
1) returns beta and stores it in the hash as a lower bound value (Sloppy, Pawny, ...);
2) returns the score backed up by the search and stores it in the hash as a lower bound value (Stockfish, Robbolito, etc);
3) There are even some programs that stores in the hash the score backed up by the search and returns beta (Danasah, Redqueen, ...).

My question is: Is there any implication by choosing any of these options?

Thanks,

Barely makes any difference. "fail soft" is the idea of returning scores < alpha or > beta. It is important in mtd(f) searches as it gives you an idea of how far to shift the bound to get closer to the true score. Otherwise, it doesn't make much if any difference at all...

With fail-soft you may know whether a move results in checkmate even with a zero-window search around a low score. What's the point of throwing away such data when it's so easy to get?

Considering that chess engines are apparently so near a local maximum that people spend time going for 1-5 elo point changes, I would think that fail soft can easily be worth it.

How often does that happen in a real game? And when it does, you discover the fact very quickly.

[rbarreira]

It might be a silly question, but there we go...
By inspecting chess programs sources, whenever occurs a beta cutoff some of them:
1) returns beta and stores it in the hash as a lower bound value (Sloppy, Pawny, ...);
2) returns the score backed up by the search and stores it in the hash as a lower bound value (Stockfish, Robbolito, etc);
3) There are even some programs that stores in the hash the score backed up by the search and returns beta (Danasah, Redqueen, ...).

My question is: Is there any implication by choosing any of these options?

Thanks,

Barely makes any difference. "fail soft" is the idea of returning scores < alpha or > beta. It is important in mtd(f) searches as it gives you an idea of how far to shift the bound to get closer to the true score. Otherwise, it doesn't make much if any difference at all...

With fail-soft you may know whether a move results in checkmate even with a zero-window search around a low score. What's the point of throwing away such data when it's so easy to get?

Considering that chess engines are apparently so near a local maximum that people spend time going for 1-5 elo point changes, I would think that fail soft can easily be worth it.

How often does that happen in a real game? And when it does, you discover the fact very quickly.

Often enough that I've seen similar stuff happening even though I don't run tests with thousands of games. And there are less extreme cases that show the value of fail-soft too, which are more frequent.

I'm a bit surprised that you would discount such an improvement, since I've often seen you say that you are interested in improvements even if they give just 1 or 2 elo points.

[bob]

It might be a silly question, but there we go...
By inspecting chess programs sources, whenever occurs a beta cutoff some of them:
1) returns beta and stores it in the hash as a lower bound value (Sloppy, Pawny, ...);
2) returns the score backed up by the search and stores it in the hash as a lower bound value (Stockfish, Robbolito, etc);
3) There are even some programs that stores in the hash the score backed up by the search and returns beta (Danasah, Redqueen, ...).

My question is: Is there any implication by choosing any of these options?

Thanks,

Barely makes any difference. "fail soft" is the idea of returning scores < alpha or > beta. It is important in mtd(f) searches as it gives you an idea of how far to shift the bound to get closer to the true score. Otherwise, it doesn't make much if any difference at all...

With fail-soft you may know whether a move results in checkmate even with a zero-window search around a low score. What's the point of throwing away such data when it's so easy to get?

Considering that chess engines are apparently so near a local maximum that people spend time going for 1-5 elo point changes, I would think that fail soft can easily be worth it.

How often does that happen in a real game? And when it does, you discover the fact very quickly.

Often enough that I've seen similar stuff happening even though I don't run tests with thousands of games. And there are less extreme cases that show the value of fail-soft too, which are more frequent.

I'm a bit surprised that you would discount such an improvement, since I've often seen you say that you are interested in improvements even if they give just 1 or 2 elo points.

The question is, where is the improvement? We get a fail high. Or a fail low. How does knowing a more accurate bound help? I don't see where this is going to improve the playing skill...

Yes, I'll _always_ take an Elo improvement, I just don't see one here.

[jwes]

The question is, where is the improvement? We get a fail high. Or a fail low. How does knowing a more accurate bound help? I don't see where this is going to improve the playing skill...

Yes, I'll _always_ take an Elo improvement, I just don't see one here.

The improvement is that generally when you get a fail-low or fail-high at the root, you don't have to do re-searches of stupid moves that are in the tt with fail soft.