First fail high drops in endgame

[bob]

Hi all,

I think I broke something in my engine, but I'm not sure what. Maybe someone has a clue.

When I go towards the endgame, the percentage first fail high drops to below 80% ( normally above 90%)

I'm not sure how to test/ where to look. I switched off bitbases (thought they were corrupt), switched off lazy eval. No improvement.

anybody had this before ?

Cheers,

Tony

This is often an effect produced by evaluation knowledge not being up to snuff. You rely on the tree search to discover things, and if the best move changes quite a bit, all the move ordering goes to pot and the FH percentage drops. I don't notice this myself, and in fact, often see the fh percentage rise significantly in endgames...

[bob]

Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.

Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony

I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.

that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...

[Jose Carlos]

Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.

Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony

I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.

that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...

I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.

[Tony]

Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.

Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue.

Tony

I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.

that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...

I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.

Yes, I see your point.

A good move only will improve the first hit rate. A good enough draft will skim off all these best moves, leaving on average worst moves.

You might be right.

Tony

[Tony]

Hi all,

I think I broke something in my engine, but I'm not sure what. Maybe someone has a clue.

When I go towards the endgame, the percentage first fail high drops to below 80% ( normally above 90%)

I'm not sure how to test/ where to look. I switched off bitbases (thought they were corrupt), switched off lazy eval. No improvement.

anybody had this before ?

Cheers,

Tony

This is often an effect produced by evaluation knowledge not being up to snuff. You rely on the tree search to discover things, and if the best move changes quite a bit, all the move ordering goes to pot and the FH percentage drops. I don't notice this myself, and in fact, often see the fh percentage rise significantly in endgames...

Could very well be. Search having to correct eval all the time...

I'll have a look at the signs of the endgame eval terms.

Recognizing a winning advantage is one thing, but then adjusting the score with "drive the king to the centre" or "get a low as possible mobility" is exactly what would create this beheaviour.

Anyway, the biggest problem has been solved, but this one might still be present.

Cheers,

Tony

[bob]

Did you ever measure this quantity before (with different result)? It seems normal to me that it is more difficult to find cut-moves in an end-game, as the information that distinguishes the winning from the losing moves is much more distant. In the middle game it is often decided by what is on the to-square, but in end-games it is what is on the square you are heading to in a number of moves.

Yes,
otoh, in the endgame, you accidently choose the right (good enough) move more often.

But in fine 70 , 78% is way too low. that should be 98% or something. (No nullmove, no lmr)
I've never seen it that low. It almost looks like a random choice. I'm overlooking something obvious, but I haven't got a clue

Tony

I think that's normal behaviour due to hashing. If you prune much more due to hashing (like in Fine 70), you have less good-and-easy moves to search that would otherwise rise the FH at fist move rate.

that's backward. The better your hashing rate, the more frequently you have a good first move (hash move)...

I didn't make my point clear. What I meant is: if you probe the hash table and find a good move, you have good chances to have a cutoff in the first move, thus improving your FH at first move rate. But if the draft in the hash node is enough (and the score is good for a cutoff) you just return without searching. This case happens a lot in positions like Fine 70, and the nodes you really need to search are those you don't have good hashing information for them.
I haven't tested it, and my reasoning might be flawed, but seems logical to me.

It sounds like you are not measuring it like everyone else is. what we are basically counting is this:

On _any_ node where I get a fail high, what percentage of the time does the fail-high happen on the first move as opposed to a later move?

So a hash hit would not affect that whatsoever...

The key is that I have to fail high at this node, and not on the first move, before it hurts the statistic. One ply below a fail-high node you don't get a fail high at all... and you don't count those at all either when gathering this statistical data.

As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).

[Jose Carlos]

It sounds like you are not measuring it like everyone else is. what we are basically counting is this:

On _any_ node where I get a fail high, what percentage of the time does the fail-high happen on the first move as opposed to a later move?

So a hash hit would not affect that whatsoever...

The key is that I have to fail high at this node, and not on the first move, before it hurts the statistic. One ply below a fail-high node you don't get a fail high at all... and you don't count those at all either when gathering this statistical data.

As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).

As far as your scenario about returning on the initial hash hit, you don't try a move, so you don't count that as a possible fail-high node and you don't pay any attention to whether the fail-high happens on the first or another move since none are searched anyway. The statistic only counts when you do try a move after failing to get a hash value that terminates the search here, and you actually try to search a move (the hash move going first of course, if there was one).[/quote]

I measure it exactly the same as you. The problem is not measure, but english (I've taken too long without speaking english so I don't remember it anymore ).
Let’s say I’m hitting 90% of fail high at the first move. Now let’s suppose that 1 out of 9 of those FH is found in the hash table with enough draft to get a cutoff. So we have 8/9 hits -> 88.8%. If it were 2, instead of 1, the resulting rate would be 7/8 -> 87.5%. And so on.
This is the effect I think might happen in positions like Fine 70. But as I said, I haven’t had time to test it. A possible test would be to disable hash table direct cutoff and see how it affects to the fh in first move rate.

[bob]

Here's my output for a 20+ second search on fine 70...

Code: Select all

              time=23.10  mat=1  n=94433259  fh=98%  nps=4.1M
              ext-> check=14.0M 1rep=591K mate=0 pp=0 reduce=62.7M/14.0M
              predicted=0  evals=58.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.10

Note my fh% is 98, which is way above normal, not below... For the initial opening position, searched for the same 23 seconds, I get:

Code: Select all

              time=23.32  mat=0  n=39344664  fh=87%  nps=1.7M
              ext-> check=881K 1rep=40K mate=379 pp=0 reduce=21.1M/3.0M
              predicted=0  evals=31.2M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.32

The fine 70 search was beyond 40 plies, the opening position was depth=15...

[Tony]

Here's my output for a 20+ second search on fine 70...

Code: Select all

              time=23.10  mat=1  n=94433259  fh=98%  nps=4.1M
              ext-> check=14.0M 1rep=591K mate=0 pp=0 reduce=62.7M/14.0M
              predicted=0  evals=58.0M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.10

Note my fh% is 98, which is way above normal, not below... For the initial opening position, searched for the same 23 seconds, I get:

Code: Select all

              time=23.32  mat=0  n=39344664  fh=87%  nps=1.7M
              ext-> check=881K 1rep=40K mate=379 pp=0 reduce=21.1M/3.0M
              predicted=0  evals=31.2M  50move=0  EGTBprobes=0  hits=0
              SMP->  splits=0  aborts=0  data=0/128  elap=23.32

The fine 70 search was beyond 40 plies, the opening position was depth=15...

That is basicly my problem now. In the opening position I get 96.6%, in Fine70 <70%

Still not fixed

Tony

[Gerd Isenberg]

That is basicly my problem now. In the opening position I get 96.6%, in Fine70 <70%

Still not fixed

Tony

And how much is fine70 without probing hashtables?
And with probing hash but using only the move?