TT Succesful hits

[bob]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.

Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).

Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.

Vincent

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

[Felpo]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.

Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).

Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.

Vincent

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Hi Dr. Hyatt,
well it is true, having the "phantom ep" is something stupid in term of TT probing, I will definitely correct this. But I had a look in the hbm code for perft, and I think I did something wrong as well in the way I store the results for further probe... I will investigate...
Just asking your opinion: as a professional developer ( not in chess for sure ) I had a good opinion in testing, automating testing: i did dicover a lot of bug in my zobrist key generation using perft as a debugging tool. Even *before* posting here. What do you think about perft as a testing in TT probing ? In my opinion, if I reach perfect move count with a good perft test suite, at a reasonable depth, I can trust my TT probing/zobrist key generation, can't I trust my TT code for the alpha beta as well ?

[bob]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.

Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).

Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.

Vincent

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Hi Dr. Hyatt,
well it is true, having the "phantom ep" is something stupid in term of TT probing, I will definitely correct this. But I had a look in the hbm code for perft, and I think I did something wrong as well in the way I store the results for further probe... I will investigate...
Just asking your opinion: as a professional developer ( not in chess for sure ) I had a good opinion in testing, automating testing: i did dicover a lot of bug in my zobrist key generation using perft as a debugging tool. Even *before* posting here. What do you think about perft as a testing in TT probing ? In my opinion, if I reach perfect move count with a good perft test suite, at a reasonable depth, I can trust my TT probing/zobrist key generation, can't I trust my TT code for the alpha beta as well ?

I suppose that is OK, but the problem is that perft won't come anywhere near proving your TT works for a normal search. You have depth issues, UPPER/LOWER/EXACT issues, replacement issues, etc. I have a -DDEBUG option in Crafty that compiles in several blocks of test code, plus a fairly large function that computes and validates all the critical data structures. It recomputes the hash/pawn-hash signatures and makes sure they agree with the current ones, it makes sure that the bitboards do not have the same bit set on any two, as that would be two different pieces on the same square, etc. It slows the code down by a factor of 10, but when I can run several tests without a single error, I can feel confident that the basic data structures are being updated properly.

[Felpo]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.

Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).

Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.

Vincent

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Hi Dr. Hyatt,
well it is true, having the "phantom ep" is something stupid in term of TT probing, I will definitely correct this. But I had a look in the hbm code for perft, and I think I did something wrong as well in the way I store the results for further probe... I will investigate...
Just asking your opinion: as a professional developer ( not in chess for sure ) I had a good opinion in testing, automating testing: i did dicover a lot of bug in my zobrist key generation using perft as a debugging tool. Even *before* posting here. What do you think about perft as a testing in TT probing ? In my opinion, if I reach perfect move count with a good perft test suite, at a reasonable depth, I can trust my TT probing/zobrist key generation, can't I trust my TT code for the alpha beta as well ?

I suppose that is OK, but the problem is that perft won't come anywhere near proving your TT works for a normal search. You have depth issues, UPPER/LOWER/EXACT issues, replacement issues, etc. I have a -DDEBUG option in Crafty that compiles in several blocks of test code, plus a fairly large function that computes and validates all the critical data structures. It recomputes the hash/pawn-hash signatures and makes sure they agree with the current ones, it makes sure that the bitboards do not have the same bit set on any two, as that would be two different pieces on the same square, etc. It slows the code down by a factor of 10, but when I can run several tests without a single error, I can feel confident that the basic data structures are being updated properly.

That is true, but starting to develop an alpha-beta + TT when TT logic is broken at the basic stages will possibly mess-up finding the solution. I will try to study what kind of test you do in Crafty in -DDEBUG

[wgarvin]

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Not only that, but (as was discussed in a different thread here a while ago) if you use the Zobrist keys for repetition detection, then a "phantom EP square" could make the repetition code think that a position was not a repeat when it actually was (or vice versa). In the interests of correctness, he should NOT put any EP square into the Zobrist key unless a pawn is in the correct place to capture it. It is the only sensible way to handle it.

[xsadar]

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Not only that, but (as was discussed in a different thread here a while ago) if you use the Zobrist keys for repetition detection, then a "phantom EP square" could make the repetition code think that a position was not a repeat when it actually was (or vice versa). In the interests of correctness, he should NOT put any EP square into the Zobrist key unless a pawn is in the correct place to capture it. It is the only sensible way to handle it.

To be completely correct, you would have to determine if the EP move leaves the mover in check. Personally, I think it's a waste of time to determine that while making the pawn advance move during a search. It won't hurt much if the engine thinks the second repetition is not a repetition if the pawn advance hasn't even been made on the board yet. However, once the move is made on the board, I agree that the zobrist key MUST be correct, which means that you not only need to make sure there is a pawn there to capture it, but also that capturing it would not put the mover in check.

Edit: Just for clarification, by 'made on the board', I mean the actual game board that the opponent sees.

[bob]

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Not only that, but (as was discussed in a different thread here a while ago) if you use the Zobrist keys for repetition detection, then a "phantom EP square" could make the repetition code think that a position was not a repeat when it actually was (or vice versa). In the interests of correctness, he should NOT put any EP square into the Zobrist key unless a pawn is in the correct place to capture it. It is the only sensible way to handle it.

To be completely correct, you would have to determine if the EP move leaves the mover in check. Personally, I think it's a waste of time to determine that while making the pawn advance move during a search. It won't hurt much if the engine thinks the second repetition is not a repetition if the pawn advance hasn't even been made on the board yet. However, once the move is made on the board, I agree that the zobrist key MUST be correct, which means that you not only need to make sure there is a pawn there to capture it, but also that capturing it would not put the mover in check.

Edit: Just for clarification, by 'made on the board', I mean the actual game board that the opponent sees.

The repetition detection is not important when you think about it. Once you push a pawn, a repetition is impossible with previous positions.

[wgarvin]

The repetition detection is not important when you think about it. Once you push a pawn, a repetition is impossible with previous positions.

The position with the EP square can't be repeated anyway. However, confusing it with a similar position (where no EP capture was possible) could lead to an engine thinking that the position had been seen 3 times when in actuality the first one was different and the latter one had only been seen 2 times.

I guess its not going to occur very often, but you still might as well avoid it if possible. I think Mike's point about the EP move not being a valid move if it leaves the mover in check is a good point also. For pseudolegal move generators, that might be a bit expensive to test! Then again, you don't need the Zobrist keys of positions that were already played over the board, for anything except maybe rep detection. So if you had the "almost-but-not-quite-correct" key during search but the "100% totally correct" key once it was over-the-board, that would probably be fine here. (To detect an *actual* repetition of a position which most engines would consider to have an EP square except for the fact of the EP move(s) being illegal moves)

[xsadar]

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Not only that, but (as was discussed in a different thread here a while ago) if you use the Zobrist keys for repetition detection, then a "phantom EP square" could make the repetition code think that a position was not a repeat when it actually was (or vice versa). In the interests of correctness, he should NOT put any EP square into the Zobrist key unless a pawn is in the correct place to capture it. It is the only sensible way to handle it.

To be completely correct, you would have to determine if the EP move leaves the mover in check. Personally, I think it's a waste of time to determine that while making the pawn advance move during a search. It won't hurt much if the engine thinks the second repetition is not a repetition if the pawn advance hasn't even been made on the board yet. However, once the move is made on the board, I agree that the zobrist key MUST be correct, which means that you not only need to make sure there is a pawn there to capture it, but also that capturing it would not put the mover in check.

Edit: Just for clarification, by 'made on the board', I mean the actual game board that the opponent sees.

The repetition detection is not important when you think about it. Once you push a pawn, a repetition is impossible with previous positions.

We're talking about repetition of a position following a two-square pawn push where en passant is not a legal move. Obviously it can't be a repetition of a previous position, but it can be repeated later. So once the pawn push has been played over the board, it's important to have the correct zobrist so we recognize if the position is repeated in the future. If the move was only played in the search, however, not having the phantom ep in the zobrist is not nearly as important, and probably not worth determining if ep is fully-legal (as opposed to only pseudo-legal) while we're making the two-square pawn move. We might as well just wait until we make the ep move (or at least generate it) to determine that.

[bob]

I do it the more correct way and do _not_ set the EP target if no EP capture is possible. Why would you want to do that since in a position where white has a pawn on the e file, and black has no pawns on the D or F files, it doesn't matter whether white plays e2-e4 or e3-e4, the two positions are _identical_. I've always done this the correct way and only set the EP target if an EP capture is actually possible, which means the Zobrist key does not get modified on e2-e4 unless black actually has a pawn on d4 or f4...

I don't see why anyone would do it any other way.

Not only that, but (as was discussed in a different thread here a while ago) if you use the Zobrist keys for repetition detection, then a "phantom EP square" could make the repetition code think that a position was not a repeat when it actually was (or vice versa). In the interests of correctness, he should NOT put any EP square into the Zobrist key unless a pawn is in the correct place to capture it. It is the only sensible way to handle it.

To be completely correct, you would have to determine if the EP move leaves the mover in check. Personally, I think it's a waste of time to determine that while making the pawn advance move during a search. It won't hurt much if the engine thinks the second repetition is not a repetition if the pawn advance hasn't even been made on the board yet. However, once the move is made on the board, I agree that the zobrist key MUST be correct, which means that you not only need to make sure there is a pawn there to capture it, but also that capturing it would not put the mover in check.

Edit: Just for clarification, by 'made on the board', I mean the actual game board that the opponent sees.

The repetition detection is not important when you think about it. Once you push a pawn, a repetition is impossible with previous positions.

We're talking about repetition of a position following a two-square pawn push where en passant is not a legal move. Obviously it can't be a repetition of a previous position, but it can be repeated later. So once the pawn push has been played over the board, it's important to have the correct zobrist so we recognize if the position is repeated in the future. If the move was only played in the search, however, not having the phantom ep in the zobrist is not nearly as important, and probably not worth determining if ep is fully-legal (as opposed to only pseudo-legal) while we're making the two-square pawn move. We might as well just wait until we make the ep move (or at least generate it) to determine that.

On the other side of the EP or not-EP move, you're right. I was thinking about positions before, but starting the new sequence off with a bogus hash signature could cause a problem. My only error in Crafty is that I do not screen for legality.