First fail high drops in endgame

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20558
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: First fail high drops in endgame

Post by bob » Tue May 29, 2007 3:10 pm

Tony wrote:Hi all,

I think I broke something in my engine, but I'm not sure what. Maybe someone has a clue.

When I go towards the endgame, the percentage first fail high drops to below 80% ( normally above 90%)

I'm not sure how to test/ where to look. I switched off bitbases (thought they were corrupt), switched off lazy eval. No improvement.

anybody had this before ?

Cheers,

Tony
This is often an effect produced by evaluation knowledge not being up to snuff. You rely on the tree search to discover things, and if the best move changes quite a bit, all the move ordering goes to pot and the FH percentage drops. I don't notice this myself, and in fact, often see the fh percentage rise significantly in endgames...

bob
Posts: 20558
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: First fail high drops in endgame

Post by bob » Tue May 29, 2007 3:12 pm

José Carlos wrote:
Tony wrote:
hgm wrote:Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.
Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony
I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.
that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...

Jose Carlos
Posts: 151
Joined: Wed Mar 08, 2006 9:09 pm
Location: Murcia (Spain)

Re: First fail high drops in endgame

Post by Jose Carlos » Wed May 30, 2007 7:48 am

bob wrote:
José Carlos wrote:
Tony wrote:
hgm wrote:Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.
Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony
I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.
that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...
I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.
__________________________
José Carlos Martínez Galán

Tony

Re: First fail high drops in endgame

Post by Tony » Wed May 30, 2007 8:15 am

José Carlos wrote:
bob wrote:
José Carlos wrote:
Tony wrote:
hgm wrote:Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.
Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony
I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.
that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...
I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.
Yes, I see your point.

A good move only will improve the first hit rate. A good enough draft will skim off all these best moves, leaving on average worst moves.

You might be right.

Tony

Tony

Re: First fail high drops in endgame

Post by Tony » Wed May 30, 2007 11:52 am

bob wrote:
Tony wrote:Hi all,

I think I broke something in my engine, but I'm not sure what. Maybe someone has a clue.

When I go towards the endgame, the percentage first fail high drops to below 80% ( normally above 90%)

I'm not sure how to test/ where to look. I switched off bitbases (thought they were corrupt), switched off lazy eval. No improvement.

anybody had this before ?

Cheers,

Tony
This is often an effect produced by evaluation knowledge not being up to snuff. You rely on the tree search to discover things, and if the best move changes quite a bit, all the move ordering goes to pot and the FH percentage drops. I don't notice this myself, and in fact, often see the fh percentage rise significantly in endgames...
Could very well be. Search having to correct eval all the time...

I'll have a look at the signs of the endgame eval terms.

Recognizing a winning advantage is one thing, but then adjusting the score with "drive the king to the centre" or "get a low as possible mobility" is exactly what would create this beheaviour.

Anyway, the biggest problem has been solved, but this one might still be present.

Cheers,

Tony

bob
Posts: 20558
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: First fail high drops in endgame

Post by bob » Thu May 31, 2007 1:51 am

José Carlos wrote:
bob wrote:
José Carlos wrote:
Tony wrote:
hgm wrote:Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.
Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue

Tony
I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.
that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...
I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.
It sounds like you are not measuring it like everyone else is. what we are basically counting is this:

On _any_ node where I get a fail high, what percentage of the time does the fail-high happen on the first move as opposed to a later move?

So a hash hit would not affect that whatsoever...

The key is that I have to fail high at this node, and not on the first move, before it hurts the statistic. One ply below a fail-high node you don't get a fail high at all... and you don't count those at all either when gathering this statistical data.

As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).

Jose Carlos
Posts: 151
Joined: Wed Mar 08, 2006 9:09 pm
Location: Murcia (Spain)

Re: First fail high drops in endgame

Post by Jose Carlos » Thu May 31, 2007 10:13 am

bob wrote: It sounds like you are not measuring it like everyone else is. what we are basically counting is this:

On _any_ node where I get a fail high, what percentage of the time does the fail-high happen on the first move as opposed to a later move?

So a hash hit would not affect that whatsoever...

The key is that I have to fail high at this node, and not on the first move, before it hurts the statistic. One ply below a fail-high node you don't get a fail high at all... and you don't count those at all either when gathering this statistical data.

As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).
As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).[/quote]

I measure it exactly the same as you. The problem is not measure, but english (I've taken too long without speaking english so I don't remember it anymore :D ).
Let’s say I’m hitting 90% of fail high at the first move. Now let’s suppose that 1 out of 9 of those FH is found in the hash table with enough draft to get a cutoff. So we have 8/9 hits -> 88.8%. If it were 2, instead of 1, the resulting rate would be 7/8 -> 87.5%. And so on.
This is the effect I think might happen in positions like Fine 70. But as I said, I haven’t had time to test it. A possible test would be to disable hash table direct cutoff and see how it affects to the fh in first move rate.
__________________________
José Carlos Martínez Galán

bob
Posts: 20558
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: First fail high drops in endgame

Post by bob » Thu May 31, 2007 4:30 pm

Here's my output for a 20+ second search on fine 70...

Code: Select all


              time=23.10  mat=1  n=94433259  fh=98%  nps=4.1M
              ext-> check=14.0M 1rep=591K mate=0 pp=0 reduce=62.7M/14.0M
              predicted=0  evals=58.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.10


Note my fh% is 98, which is way above normal, not below... For the initial opening position, searched for the same 23 seconds, I get:

Code: Select all

              time=23.32  mat=0  n=39344664  fh=87%  nps=1.7M
              ext-> check=881K 1rep=40K mate=379 pp=0 reduce=21.1M/3.0M
              predicted=0  evals=31.2M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.32


The fine 70 search was beyond 40 plies, the opening position was depth=15...

Tony

Re: First fail high drops in endgame

Post by Tony » Thu May 31, 2007 7:16 pm

bob wrote:Here's my output for a 20+ second search on fine 70...

Code: Select all


              time=23.10  mat=1  n=94433259  fh=98%  nps=4.1M
              ext-> check=14.0M 1rep=591K mate=0 pp=0 reduce=62.7M/14.0M
              predicted=0  evals=58.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.10


Note my fh% is 98, which is way above normal, not below... For the initial opening position, searched for the same 23 seconds, I get:

Code: Select all

              time=23.32  mat=0  n=39344664  fh=87%  nps=1.7M
              ext-> check=881K 1rep=40K mate=379 pp=0 reduce=21.1M/3.0M
              predicted=0  evals=31.2M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.32


The fine 70 search was beyond 40 plies, the opening position was depth=15...
That is basicly my problem now. In the opening position I get 96.6%, in Fine70 <70%

Still not fixed :(

Tony

Gerd Isenberg
Posts: 2127
Joined: Wed Mar 08, 2006 7:47 pm
Location: Hattingen, Germany

Re: First fail high drops in endgame

Post by Gerd Isenberg » Thu May 31, 2007 7:58 pm

Tony wrote: That is basicly my problem now. In the opening position I get 96.6%, in Fine70 <70%

Still not fixed :(

Tony
And how much is fine70 without probing hashtables?
And with probing hash but using only the move?

Post Reply