fail soft vs fail hard

[bob]

Although this might be true, it seems to me that this kind of simplification would be infinitesimal on the scale of any program that is significantly larger than micro-Max. Which, btw, uses fail soft...

The gains in terms of simplicity that you would reap by not using PVS, but simple alpha-beta, is far larger.

yes, but PVS is a significant win, compared to fail-soft, which is essentially no change once you use PVS.

[bob]

But if you do fail hard in qsearch(), you are going to return a score in the range of [alpha, beta]. And then, doing fail soft in search() would be the same as fail hard, or am i missing something?

If you don't do it everywhere, you don't do it anywhere. The only real exception is in egtb scores and draws by repetition/50-move/etc... There, even in search, if you fail-soft you have a score outside A-B window to return. But for normal positions, you are correct that fail-soft and fail-hard are the same if you do fail-hard in the q-search as there is no way to get a score outside A-B.

[michiguel]

But if you do fail hard in qsearch(), you are going to return a score in the range of [alpha, beta]. And then, doing fail soft in search() would be the same as fail hard, or am i missing something?

I was doing fail soft everywhere, but I changed it in qsearch(), which is the place that gave me trouble (at least in a set of positions). I did not change it in the regular search because of laziness , not because of some rational explanation (and maybe is not a problem for me in the regular search). I need to test the whole thing, though. What you say is true, I think, if the search went in qsearch(). There are many situations where you return a score in the tree that did not touch qsearch(). For instance, checkmates, draw by repetitions, hashtable hits, egtb hits etc.

Miguel

[hgm]

yes, but PVS is a significant win, compared to fail-soft, which is essentially no change once you use PVS.

Well, PVS only starts to become a win when your move ordering is good enough (but not too good). For micro-Max it was a loss.

[Pablo Vazquez]

I see, thx for the explanation.

[jwes]

I think failing soft is always superior from a performance point of view. (If you use the scores in the hash table for cutoffs, that is. Otherwise it is strictly equivalent.)

The only advantage I see is that with fail hard you will catch any bugs in your alpha-beta implementation immediately, as any cutoff will only be marginal: you will never have scores above beta, the cutoffs all occur by a score equal to beta. This can be important in implementations where setting the search window is not trivial. Such as what I do when using a delayed-loss bonus.

I would add that it probably depends on your basic search approach. If you use PVS, so that almost 100% of the searches are with a null-window, fail-soft offers nothing over fail-hard. The only case I have found where fail-soft is important is in mtd(f) where fail-soft gives you a decent "hint" about how far off the root null-window actually is, so that you can converge faster.

I have tried both in Crafty and found no difference that could be measured... But then I have been using PVS since around 1980 or so...

Could you put fail soft in Crafty as a compiler option so we could play with it ?

[cyberfish]

Ah I see. Thanks.

I just tried it with my engine, and noticed no difference, too.

[Sven]

I would add that it probably depends on your basic search approach.

That is also my opinion. I see the following different approaches of doing the search, where "plain alpha-beta" is meant as either fail-hard or fail-soft:

1. plain alpha-beta without aspiration window
=> without TT hash, fail-soft and fail-hard are equivalent, the additional local variable does not affect performance noticeably
=> with TT hash, there may be a small advantage for fail-soft since storing slightly better bounds may cause slightly more hash hits (I remember having read something about 7% but I can't find the source for it anymore, are there any measurements?)

2. plain alpha-beta with aspiration window
=> same difference as 1. regarding hash hits
=> additionally, each time when a search fails low or high *at search root* the re-search will often require less nodes with fail-soft than with fail-hard since the window for re-search is smaller than with fail-hard and produces slightly more cutoffs

3. PVS without aspiration window
=> should be the same as 1., so "only" probably more hash hits
=> documents describing PVS algorithm nearly always use fail-soft instead of fail-hard; that does not mean that fail-soft is really any better in a PVS framework than fail-hard, it could also mean that both are equivalent here

4. PVS with aspiration window (I don't know whether many engines are using this approach)
=> should be the same as 2., so probably more hash hits and slightly faster re-search at root in case of failing low or high

5. MTD(f)
=> as far as I read fail-soft is significantly better here than fail-hard


The remaining question for me is: what is the gain of fail-soft over fail-hard regarding number of hash hits, as mentioned in 1. above? A plain alpha-beta search *without* aspiration window might be used to compare both variants, preferably over many games, and watching out for both hash hit statistics and playing strength. If there is a gain, it should also exist when using an aspiration window and/or PVS. Has anybody done such a test already and could post results?

Sven

[AndrewShort]

I would add that it probably depends on your basic search approach.

That is also my opinion.
4. PVS with aspiration window (I don't know whether many engines are using this approach)

Sven

I use fail hard,pvs,and aspiration windows. Is there something inherently unsound in using pvs and aspiration windows?

[Sven]

I would add that it probably depends on your basic search approach.

That is also my opinion.
4. PVS with aspiration window (I don't know whether many engines are using this approach)

Sven

I use fail hard,pvs,and aspiration windows. Is there something inherently unsound in using pvs and aspiration windows?

At least in this thread: http://www.talkchess.com/forum/viewtopic.php?p=197564 there were some pros and cons.

Sven