size of hash tables engines need at bullet time control?

[Uri Blass]

If 512 mbytes is significantly stronger than 64 mbytes for stockfish at bullet then I find it to be surprising(and note that VVLTC of 70+0.7 with 7 threads is still bullet time control).

Author: Viren6
Date: Sun May 19 09:37:22 2024 +0200
Timestamp: 1716104242

Revert "Simplify Away Quadruple Extensions"

This reverts commit 4edd1a3

The unusual result of (combined) +12.0 +- 3.7 in the 2 VVLTC simplification SPRTs ran was the result of base having only 64MB of hash instead of 512MB (Asymmetric hash).
Vizvezdenec was the one to notice this.

closes https://github.com/official-stockfish/S ... /pull/5265

bench 1404295

[Jouni]

SF wiki has some hash tests. At VLTC (240+2.4s):

Code: Select all

Hash 	Hashfull 	Elo 	Elo-err
256 	131 	0.00 	0.00
128 	239 	-1.00 	7.50
64 	397 	-0.80 	6.60
32 	591 	-12.10 	6.10
16 	766 	-21.40 	7.30
8 	865 	-32.30 	4.20
4 	931 	-52.40 	6.20
2 	943 	-67.40 	5.70
1 	947 	-95.20 	6.60

Obviously too small hash can explain +12 Elo?

[hgm]

It seems a good example for how testing at unrealistically large hash size tends to accept patches that make the engine weaker at hash sizes of practical interest.

[Eelco de Groot]

If the patch was removed and at the same time the hash was lowered from 512 Mb to 64 Mb, then they were testing two things at the same time. It can not be concluded from the positive result of the test that the patch could be simplified away when in fact it passed an earlier test with 512 Mb, under I think stricter conditions even, but I'm not sure. You also can not conclude that 64 Mb is better than 512 Mb. The simplification allows a small Elo decrease as far as I remember (so less strict conditions for the removal of code to pass)? It is possible the Stockfish team or the original author of the simplification test will now rerun the test with 512 Mb, I have not checked that. I'm not sure this was a serious technical point also it's entirely possible that is not what Uri meant though and he just wondered about the hash table size. Just for the record Maybe I misunderstood I just react from memory. So don't mind me!

[JacquesRW]

It seems a good example for how testing at unrealistically large hash size tends to accept patches that make the engine weaker at hash sizes of practical interest.

How so? Doesn't it just suggest that increasing hash size from 64 to 512 gains elo under the given testing conditions?

[hgm]

It would be very sad indeed if increasing hash size would decrease Elo...

The point is that a patch was accepted with one hash size, but is rejected at another hash size.

[JacquesRW]

The point is that a patch was accepted with one hash size, but is rejected at another hash size.

Except it wasn't. A patch was accepted when the dev engine had a larger hash size than the base engine. You seem to have misread that the original test used 64mb hash for both engines, when it did not.

[hgm]

Oh, I see. This is what they mean by 'asymmetric hash', that the versions playing against each other did not use the same hash size ('hash-size odds'). I had indeed not understood that.