Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):
Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?
Felpo wrote:Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):
Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?
With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.
Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).
Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.
Note that qperft does not store d=1 nodes in the main hash, though, but is a separate small hash table (similar to an eval cache), which very quickly overflows. When I store all d>0 nodes in the main hash I get:
hgm wrote:Note that qperft does not store d=1 nodes in the main hash, though, but is a separate small hash table (similar to an eval cache), which very quickly overflows. When I store all d>0 nodes in the main hash I get:
Thanks for showing me some results. My version of your qperft does not shown these. I've no idea on why my numbers are differents. But now I have a target to compare to.
Felpo wrote:My version of your qperft does not shown these. I've no idea on why my numbers are differents.
I just put in the counters and print statements for the occasion.
There can be many reasons why the numbers are different. For one you should print total number of probes as wel as hits, to have a better idea of the hit rate. You will do more probes as I do, because of the phantom e.p. rights. This makes your tree larger because you are not getting hits where you should have gotten them. That means you get more hits even if the hit rate is lower.
Even without this, the exact assignment of Zobrist keys causes cance fluctuations, which can produce large difference in an always-replace scheme. (If you are so unlucky that an entry close to the root is overwritten.)
diep wrote:With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines.
This is faulty match, Vincent. The hit rate is far higher. The perft count is not equal to the number of nodes visited (and thus number of probes). The latter is much smaller, exactly because there are hash hits that prune the tree.
diep wrote:With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines.
This is faulty match, Vincent. The hit rate is far higher. The perft count is not equal to the number of nodes visited (and thus number of probes). The latter is much smaller, exactly because there are hash hits that prune the tree.
Ah yes i see your point, it is a valid one here.
Of course the point still stands that a specialized hashtable for perft is total different from a hashtable for a normal chess engine.
So any statement that perft gets used to 'test' the hashtable implementation is 100% ridicilous nonsense.
Even testing the number of bits for zobrist it will do wrong. You can easily make it clear (proving is tougher) that you can get away with a smaller total keysize for perft than in a chessprogram, provided you want to search with 0 errors in both cases.
If you erroneously update your hash key, you will get many false 'hits', which will screw up the perft count. The fact that you see correct counts is very strong evidence that you retrieve what you stored, and that you retreive what you tried to retreive, i.e. that your hash table indeed works as a hash table and not as an elaborate scheme for random generation.
So indeed, it is a very useful test. There is nothing specialized to perft in these hash tables. Hashing for perft and search use the same hash key.