TT Succesful hits

[Felpo]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

[diep]

Hi all,
I did some statistics with my TT subsystem to check how many time a succesful trasposition is found during perft. Below the numbers ( initial position ):

Code: Select all

Depth	Moves		Hits
====================================
4	197281		1298
5 	4865609	 	68219
6 	119060324 	1400809
7 	3195901860 	25747643
8	84998978956 	756243348
9	2439530234167 	19122021983

Please consider that en passant square is considered in the zobrist key even if no pawns can capture it. Entries count is 2^21, replacement policy is newest replace the older always.
Succesful hits happens only from depth 4.
Are these numbers consistent in your opinion/experience ?

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines. In normal chess engines in opening/middlegame typical hitrate is 4% to 5% and in endgames it goes up quite a tad to 20% in far (pawn)endgames.

Of course a tad more hits are succesful than that just for the bestmove and in about 10% of the cases i get the evaluation out of the TT (not to confuse with the evaluation hashtable which is another giant hashtable that i also use for this purpose as well).

Also of course what works well for perft will be different for normal chess. You can optimize those TT's a lot for perft to work better for it (storing several depths for example in the TT, whereas in chess you prefer the biggest depth of course), it makes making a special optimized perft searcher seem even more ridicioulous.

Vincent

[hgm]

Some stats from qperft:

Code: Select all

perft(1)=20 ( 0.000 sec)        probes=       0, entries=       0 hits=       0
perft(2)=400 ( 0.000 sec)       probes=      20, entries=      20 hits=       0
perft(3)=8902 ( 0.000 sec)      probes=     420, entries=     436 hits=       0
perft(4)=197281 ( 0.030 sec)    probes=    9322, entries=    5298 hits=    3041
perft(5)=4865609 ( 0.180 sec)   probes=  129055, entries=   22436 hits=   44271
perft(6)=119060324 ( 2.904 sec) probes= 1948836, entries=   97107 hits=  520739

This is with 32MB hash (2M entries). It is treating e.p. rights in the correct way, though.

[hgm]

Note that qperft does not store d=1 nodes in the main hash, though, but is a separate small hash table (similar to an eval cache), which very quickly overflows. When I store all d>0 nodes in the main hash I get:

Code: Select all

perft(1)=20 ( 0.000 sec)        probes=       0, entries=       0 hits=       0
perft(2)=400 ( 0.000 sec)       probes=      20, entries=      20 hits=       0
perft(3)=8902 ( 0.000 sec)      probes=     420, entries=     438 hits=       0
perft(4)=197281 ( 0.030 sec)    probes=    9322, entries=    6164 hits=    3490
perft(5)=4865609 ( 0.180 sec)   probes=  129055, entries=   80700 hits=   50152
perft(6)=119060324 ( 2.523 sec) probes= 1984149, entries=  677930 hits=  928486

[Felpo]

Note that qperft does not store d=1 nodes in the main hash, though, but is a separate small hash table (similar to an eval cache), which very quickly overflows. When I store all d>0 nodes in the main hash I get:

Code: Select all

perft(1)=20 ( 0.000 sec)        probes=       0, entries=       0 hits=       0
perft(2)=400 ( 0.000 sec)       probes=      20, entries=      20 hits=       0
perft(3)=8902 ( 0.000 sec)      probes=     420, entries=     438 hits=       0
perft(4)=197281 ( 0.030 sec)    probes=    9322, entries=    6164 hits=    3490
perft(5)=4865609 ( 0.180 sec)   probes=  129055, entries=   80700 hits=   50152
perft(6)=119060324 ( 2.523 sec) probes= 1984149, entries=  677930 hits=  928486

Thanks for showing me some results. My version of your qperft does not shown these. I've no idea on why my numbers are differents. But now I have a target to compare to.

[hgm]

My version of your qperft does not shown these. I've no idea on why my numbers are differents.

I just put in the counters and print statements for the occasion.

There can be many reasons why the numbers are different. For one you should print total number of probes as wel as hits, to have a better idea of the hit rate. You will do more probes as I do, because of the phantom e.p. rights. This makes your tree larger because you are not getting hits where you should have gotten them. That means you get more hits even if the hit rate is lower.

Even without this, the exact assignment of Zobrist keys causes cance fluctuations, which can produce large difference in an always-replace scheme. (If you are so unlucky that an entry close to the root is overwritten.)

[hgm]

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines.

This is faulty match, Vincent. The hit rate is far higher. The perft count is not equal to the number of nodes visited (and thus number of probes). The latter is much smaller, exactly because there are hash hits that prune the tree.

[diep]

With that 0.7% hitrate perft total disqualifies itself again to compare to normal chess engines.

This is faulty match, Vincent. The hit rate is far higher. The perft count is not equal to the number of nodes visited (and thus number of probes). The latter is much smaller, exactly because there are hash hits that prune the tree.

Ah yes i see your point, it is a valid one here.

Of course the point still stands that a specialized hashtable for perft is total different from a hashtable for a normal chess engine.

So any statement that perft gets used to 'test' the hashtable implementation is 100% ridicilous nonsense.

Even testing the number of bits for zobrist it will do wrong. You can easily make it clear (proving is tougher) that you can get away with a smaller total keysize for perft than in a chessprogram, provided you want to search with 0 errors in both cases.

[hgm]

If you erroneously update your hash key, you will get many false 'hits', which will screw up the perft count. The fact that you see correct counts is very strong evidence that you retrieve what you stored, and that you retreive what you tried to retreive, i.e. that your hash table indeed works as a hash table and not as an elaborate scheme for random generation.

So indeed, it is a very useful test. There is nothing specialized to perft in these hash tables. Hashing for perft and search use the same hash key.

[Felpo]

And, testing in perft is a lot easier than testing in search...