Return eval or upper bound?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Return eval or upper bound?

Post by bob »

JVMerlino wrote:
bob wrote:
hgm wrote:Both work. The first is known as fail-hard, the other as fail-soft. When you use a hash table, fail-soft is more efficient, as it tehds to lead to tighter bounds on the moves, which then get stored in tha hash table.
Also adds a bit of instability, although I still use it. unstable != bad, just a pain at times when debugging or testing.
Ok, that's what I thought, as I'm occasionally seeing very strange (although very temporary) PVs. So I guess that's what happens with fail-soft when it affects a move at the root?

The reason I caught this is in this position from the WAC (#264):

r2r2k1/1R2qp2/p5pp/2P5/b1PN1b2/P7/1Q3PPP/1B1R2K1 b - -

Myrddin's output through the first half-second with fail-soft was this:

Code: Select all

 
 1/ 4  -146      6        17  f4h2 g1h2 a4d1 
 2/ 7  -106      6       447  f4h2? g1h2 
 2/ 8    79      7      1932  f4h2 g1h1 h2c7 h1h2 
 2/ 8    60      7      3054  e7c5 b1c2 c5c4 
 3/ 8    65      7      3945  e7c5 b1c2 c5c4 c2a4 c4a4 
 3/10    20      9      4391  f4h2! b1c2 c5c4 c2a4 c4a4 
 3/10   -81      9      6982  f4h2 g1h1 e7b7 b2b7 a4d1 
 4/16  -109     12     26571  f4h2 g1h1 e7h4 b1c2 h2g3 h1g1 g3f2 g1f1 f2d4 c2a4 
 4/16  -116     14     31455  d8d4 b2d4 e7b7 d1e1 g8h8 
 4/16  -121     15     37123  e7c5! d1e1 
 4/16  -191     17     43245  e7c5 b7f7 g8f7 b2b7 f4c7 
 4/16 -32764     21     69743  d8d4 b7e7 
 5/10 -32724     21     72173  d8d4? b2d4 e7e1 g1h2 
 5/16  -116     32    151759  d8d4 b2d4 e7b7 d1e1 b7c7 
 5/16  -119     39    184296  f4h2 g1h2 e7e5 f2f4 e5h5 h2g3 h5d1 d4f3 
 5/16  -173     43    209067  e7e5 b1c2 f4h2 g1h1 a4c2 b2c2 d8d4 
Obviously, that mate in 2 score for Black is just dead wrong.

So I changed it to fail-hard, and it doesn't happen any more.

Or would the above be caused by something else entirely?

Thanks to all who have contributed!

jm
The most notable thing I see, and not all that often, is a fail high, but then the resulting search fails low and you get no PV, or a partial PV. With normal PVS, and fail-hard, you simply get a ton of alpha, alpha+1 bound position entries in the hash table. If you shift the window slightly, none of those entries will help find a PV if the new bounds are slightly higher. But with fail-soft, you can still find a bount stored that is above any modified bound you use, and fail high. And then on the re-search you can't see deeply enough to get a real score.

The fail high is actually correct, but it looks odd to not get a PV back.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Return eval or upper bound?

Post by bob »

bob wrote:
JVMerlino wrote:
bob wrote:
hgm wrote:Both work. The first is known as fail-hard, the other as fail-soft. When you use a hash table, fail-soft is more efficient, as it tehds to lead to tighter bounds on the moves, which then get stored in tha hash table.
Also adds a bit of instability, although I still use it. unstable != bad, just a pain at times when debugging or testing.
Ok, that's what I thought, as I'm occasionally seeing very strange (although very temporary) PVs. So I guess that's what happens with fail-soft when it affects a move at the root?

The reason I caught this is in this position from the WAC (#264):

r2r2k1/1R2qp2/p5pp/2P5/b1PN1b2/P7/1Q3PPP/1B1R2K1 b - -

Myrddin's output through the first half-second with fail-soft was this:

Code: Select all

 
 1/ 4  -146      6        17  f4h2 g1h2 a4d1 
 2/ 7  -106      6       447  f4h2? g1h2 
 2/ 8    79      7      1932  f4h2 g1h1 h2c7 h1h2 
 2/ 8    60      7      3054  e7c5 b1c2 c5c4 
 3/ 8    65      7      3945  e7c5 b1c2 c5c4 c2a4 c4a4 
 3/10    20      9      4391  f4h2! b1c2 c5c4 c2a4 c4a4 
 3/10   -81      9      6982  f4h2 g1h1 e7b7 b2b7 a4d1 
 4/16  -109     12     26571  f4h2 g1h1 e7h4 b1c2 h2g3 h1g1 g3f2 g1f1 f2d4 c2a4 
 4/16  -116     14     31455  d8d4 b2d4 e7b7 d1e1 g8h8 
 4/16  -121     15     37123  e7c5! d1e1 
 4/16  -191     17     43245  e7c5 b7f7 g8f7 b2b7 f4c7 
 4/16 -32764     21     69743  d8d4 b7e7 
 5/10 -32724     21     72173  d8d4? b2d4 e7e1 g1h2 
 5/16  -116     32    151759  d8d4 b2d4 e7b7 d1e1 b7c7 
 5/16  -119     39    184296  f4h2 g1h2 e7e5 f2f4 e5h5 h2g3 h5d1 d4f3 
 5/16  -173     43    209067  e7e5 b1c2 f4h2 g1h1 a4c2 b2c2 d8d4 
Obviously, that mate in 2 score for Black is just dead wrong.

So I changed it to fail-hard, and it doesn't happen any more.

Or would the above be caused by something else entirely?

Thanks to all who have contributed!

jm
The most notable thing I see, and not all that often, is a fail high, but then the resulting search fails low and you get no PV, or a partial PV. With normal PVS, and fail-hard, you simply get a ton of alpha, alpha+1 bound position entries in the hash table. If you shift the window slightly, none of those entries will help find a PV if the new bounds are slightly higher. But with fail-soft, you can still find a bount stored that is above any modified bound you use, and fail high. And then on the re-search you can't see deeply enough to get a real score.

The fail high is actually correct, but it looks odd to not get a PV back.
You should not see wrong scores, ever, unless they are the result of your program simply missing something due to lack of depth. If you produce a mate score it should never be wrong. You _might_ produce a mate score from a hash hit, and then lose it later if you happen to overwrite the critical hash entry. You should pick it back up once you get deep enough to see the mate in your search, although that is not guaranteed to happen.
JVMerlino
Posts: 1357
Joined: Wed Mar 08, 2006 10:15 pm
Location: San Francisco, California

Re: Return eval or upper bound?

Post by JVMerlino »

bob wrote:
JVMerlino wrote:
bob wrote:
hgm wrote:Both work. The first is known as fail-hard, the other as fail-soft. When you use a hash table, fail-soft is more efficient, as it tehds to lead to tighter bounds on the moves, which then get stored in tha hash table.
Also adds a bit of instability, although I still use it. unstable != bad, just a pain at times when debugging or testing.
Ok, that's what I thought, as I'm occasionally seeing very strange (although very temporary) PVs. So I guess that's what happens with fail-soft when it affects a move at the root?

The reason I caught this is in this position from the WAC (#264):

r2r2k1/1R2qp2/p5pp/2P5/b1PN1b2/P7/1Q3PPP/1B1R2K1 b - -

Myrddin's output through the first half-second with fail-soft was this:

Code: Select all

 
 1/ 4  -146      6        17  f4h2 g1h2 a4d1 
 2/ 7  -106      6       447  f4h2? g1h2 
 2/ 8    79      7      1932  f4h2 g1h1 h2c7 h1h2 
 2/ 8    60      7      3054  e7c5 b1c2 c5c4 
 3/ 8    65      7      3945  e7c5 b1c2 c5c4 c2a4 c4a4 
 3/10    20      9      4391  f4h2! b1c2 c5c4 c2a4 c4a4 
 3/10   -81      9      6982  f4h2 g1h1 e7b7 b2b7 a4d1 
 4/16  -109     12     26571  f4h2 g1h1 e7h4 b1c2 h2g3 h1g1 g3f2 g1f1 f2d4 c2a4 
 4/16  -116     14     31455  d8d4 b2d4 e7b7 d1e1 g8h8 
 4/16  -121     15     37123  e7c5! d1e1 
 4/16  -191     17     43245  e7c5 b7f7 g8f7 b2b7 f4c7 
 4/16 -32764     21     69743  d8d4 b7e7 
 5/10 -32724     21     72173  d8d4? b2d4 e7e1 g1h2 
 5/16  -116     32    151759  d8d4 b2d4 e7b7 d1e1 b7c7 
 5/16  -119     39    184296  f4h2 g1h2 e7e5 f2f4 e5h5 h2g3 h5d1 d4f3 
 5/16  -173     43    209067  e7e5 b1c2 f4h2 g1h1 a4c2 b2c2 d8d4 
Obviously, that mate in 2 score for Black is just dead wrong.

So I changed it to fail-hard, and it doesn't happen any more.

Or would the above be caused by something else entirely?

Thanks to all who have contributed!

jm
The most notable thing I see, and not all that often, is a fail high, but then the resulting search fails low and you get no PV, or a partial PV. With normal PVS, and fail-hard, you simply get a ton of alpha, alpha+1 bound position entries in the hash table. If you shift the window slightly, none of those entries will help find a PV if the new bounds are slightly higher. But with fail-soft, you can still find a bount stored that is above any modified bound you use, and fail high. And then on the re-search you can't see deeply enough to get a real score.

The fail high is actually correct, but it looks odd to not get a PV back.
The lack of a (complete) PV is probably due to a hash hit. I do not walk the hash table to derive the full PV.

jm
User avatar
hgm
Posts: 27793
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Return eval or upper bound?

Post by hgm »

That means you must have another bug, which is partly masked by using fail-hard.

If that mate score occurs reproducibly, you should not have much difficulty debugging where it came from, at such low depth. If that score comes from a hash hit, just let it print the address of the hash entry it used, and then put in code to dump the search tree that filled that hash entry to get the rest of the PV.
Gian-Carlo Pascutto
Posts: 1243
Joined: Sat Dec 13, 2008 7:00 pm

Re: Return eval or upper bound?

Post by Gian-Carlo Pascutto »

JVMerlino wrote: Ok, that's what I thought, as I'm occasionally seeing very strange (although very temporary) PVs. So I guess that's what happens with fail-soft when it affects a move at the root?
No.
Or would the above be caused by something else entirely?
Yes.