size of hash tables engines need at bullet time control?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Uri Blass
Posts: 10420
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

size of hash tables engines need at bullet time control?

Post by Uri Blass »

If 512 mbytes is significantly stronger than 64 mbytes for stockfish at bullet then I find it to be surprising(and note that VVLTC of 70+0.7 with 7 threads is still bullet time control).

Author: Viren6
Date: Sun May 19 09:37:22 2024 +0200
Timestamp: 1716104242

Revert "Simplify Away Quadruple Extensions"

This reverts commit 4edd1a3

The unusual result of (combined) +12.0 +- 3.7 in the 2 VVLTC simplification SPRTs ran was the result of base having only 64MB of hash instead of 512MB (Asymmetric hash).
Vizvezdenec was the one to notice this.

closes https://github.com/official-stockfish/S ... /pull/5265

bench 1404295
Jouni
Posts: 3324
Joined: Wed Mar 08, 2006 8:15 pm

Re: size of hash tables engines need at bullet time control?

Post by Jouni »

SF wiki has some hash tests. At VLTC (240+2.4s):

Code: Select all

Hash 	Hashfull 	Elo 	Elo-err
256 	131 	0.00 	0.00
128 	239 	-1.00 	7.50
64 	397 	-0.80 	6.60
32 	591 	-12.10 	6.10
16 	766 	-21.40 	7.30
8 	865 	-32.30 	4.20
4 	931 	-52.40 	6.20
2 	943 	-67.40 	5.70
1 	947 	-95.20 	6.60
Obviously too small hash can explain +12 Elo?
Jouni
User avatar
hgm
Posts: 27896
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: size of hash tables engines need at bullet time control?

Post by hgm »

It seems a good example for how testing at unrealistically large hash size tends to accept patches that make the engine weaker at hash sizes of practical interest.
User avatar
Eelco de Groot
Posts: 4591
Joined: Sun Mar 12, 2006 2:40 am
Full name:   

Re: size of hash tables engines need at bullet time control?

Post by Eelco de Groot »

If the patch was removed and at the same time the hash was lowered from 512 Mb to 64 Mb, then they were testing two things at the same time. It can not be concluded from the positive result of the test that the patch could be simplified away when in fact it passed an earlier test with 512 Mb, under I think stricter conditions even, but I'm not sure. You also can not conclude that 64 Mb is better than 512 Mb. The simplification allows a small Elo decrease as far as I remember (so less strict conditions for the removal of code to pass)? It is possible the Stockfish team or the original author of the simplification test will now rerun the test with 512 Mb, I have not checked that. I'm not sure this was a serious technical point also it's entirely possible that is not what Uri meant though and he just wondered about the hash table size. Just for the record 8-) Maybe I misunderstood I just react from memory. So don't mind me!
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
JacquesRW
Posts: 69
Joined: Sat Jul 30, 2022 12:12 pm
Full name: Jamie Whiting

Re: size of hash tables engines need at bullet time control?

Post by JacquesRW »

hgm wrote: Thu May 23, 2024 8:15 am It seems a good example for how testing at unrealistically large hash size tends to accept patches that make the engine weaker at hash sizes of practical interest.
How so? Doesn't it just suggest that increasing hash size from 64 to 512 gains elo under the given testing conditions?
User avatar
hgm
Posts: 27896
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: size of hash tables engines need at bullet time control?

Post by hgm »

It would be very sad indeed if increasing hash size would decrease Elo...

The point is that a patch was accepted with one hash size, but is rejected at another hash size.
JacquesRW
Posts: 69
Joined: Sat Jul 30, 2022 12:12 pm
Full name: Jamie Whiting

Re: size of hash tables engines need at bullet time control?

Post by JacquesRW »

hgm wrote: Thu May 23, 2024 10:39 pm The point is that a patch was accepted with one hash size, but is rejected at another hash size.
Except it wasn't. A patch was accepted when the dev engine had a larger hash size than the base engine. You seem to have misread that the original test used 64mb hash for both engines, when it did not.
User avatar
hgm
Posts: 27896
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: size of hash tables engines need at bullet time control?

Post by hgm »

Oh, I see. This is what they mean by 'asymmetric hash', that the versions playing against each other did not use the same hash size ('hash-size odds'). I had indeed not understood that.