Hash for engine matches

[Kyodai]

Somebody just mailed me this posted in another forum.

Thoughts? Interesting the he concludes results dramatically different



"Hash Table Size for Engine Matches
I am changing my hash table formula's for engine matches. As time has gone by and hardware and software changes have taken place ( have gotten better and better) in today's environment it is clear that more hash can and is being accessed than by the old formula's that used to be used.
The newer rule of thumb that I will now use is simply if the hard drive is not grinding away from using too much than you are fine.
Here is a reference from Steve Lopez about this topic,it is 8 years old, so it is even more true today.
http://www.chessexchange.com/forum/v...php?f=7&t=1397

I just doubled hash in 1 minute games from 256 to 512 and the results were dramatically different. You can almost tell the engines hunger for more ram, and this is on my slow i3 dual core system. I will also increase Little Blitzer hash proportionately and watch the differences."

[RJN]

Houdini has a hash size formula, which can be found online in the documentation FAQ:

http://www.cruxis.com/chess/manual/index.html

My preference is to follow the Houdini formula for my engines, then go to the next higher binary size. It's easy to remember, take kN/s, divide by 100, then multiply by average seconds per move to get the size in MB.

In my 40 in 5 minutes (repeating) tournament currently running, I use 512 MB hash, on an i5 dual core getting about 3500 kN/s.


Komodo 5.1 MP makes the following recommendations about setting hash size:

(I assume the Komodo authors don't mind me posting the below)


How should I set the hash table size?
=====================================

In general the default hash table size setting in Komodo will work
well for you. However, if you want optimal performance you may wish
to tweak this value. We are providing some guidelines here.

The optimal hash table size setting for Komodo is when the program
utilizes half, or less, of the hash table. When the percentage
exceeds 50% you will see a decline in performance.

In modern chess programs the hash tables can fill very quickly, so it
may not be possible to provide the optimal hash table size, especially
when doing deep searches or using the program for deep analysis or
where the amount of memory on a given machine is limited. So the rule
of thumb in these cases is to set the hash table size as large as
reasonable possible without sacrificing too much system performance.
Keep in mind that very heavy memory usage can impact the speed of the
program and the underlying operating system.

We have devised a general rule of thumb for determining how large to
set the hash table based on the time control you wish to play.

We consider sudden death or Fischer time controls but these guidelines
can be extrapolated to other time controls. There are system
considerations too so this is not a hard rule but really just a
general guideline and does not take into consideration the memory
caching performance of your machine. Nevertheless, most modern PC's
will do well with these settings:

1. Take the main time in minutes and add the increment in seconds
(for sudden death the increment is zero.)

2. Multiply the value obtained in step 1 by 3.

3. From the opening position, do a fixed time search of exactly this
amount of time in seconds.

4. Note the hash table utilization as reported by the GUI.

5. If the hash table utilization is much above 40%, double the hash
table size and repeat this test.

6. If the hash table utilization is too small, for example well
below 20%, you are probably setting your table higher than it
needs to be.

For technical reasons, setting a hash table that is much larger than
it needs to be will have a negative impact on the performance of your
chess program (and the rest of the system) as it place more demands on
the memory sub-system and cache. However the impact is generally
pretty minor up to some point. In general, all other things being
equal, you should error on the side of being too large rather than
being too small. We generally suggesting choosing the largest hash
table size that give no more than about 40 or 50% utilization using
this test.

[kranium]

In my 40 in 5 minutes (repeating) tournament currently running, I use 512 MB hash, on an i5 dual core getting about 3500 kN/s.

Hi Rob-

i posted a short guide on chesslogik a few years ago:
http://users.telenet.be/chesslogik/dynamic_hash.htm

basically, if you know the engine's hash entry size, match TC, and the system's NPS you can calculate RAM needed for an average move it with some precision:

40 in 5 minutes = 7.5 secs per move
typical hash entry = 16 bytes
3500 kN/s = 3.5 MN/s

optimal hash size = (time per move) x (nodes per sec) x (hash entry size) = 7.5 * 3.5 * 16 = 420 MB

so, 512 seems an ideal choice here
Norm

[RJN]

Thanks Norman, that's even better!

BTW, glad to get some confirmation that a tournament going on for 2 months isn't fundamentally flawed

[Kohflote]

Hi all,

Thank you for the info, very useful .

Please correct me if I am wrong, I guess the calculation is not applicable if time per move is long, e.g. 15 mins per move.

Best wishes,
Koh, Kah Huat

[Tomcass]

Thank you very much Norman. If I apply your suggested formulae I have to set 512 hash in my tests instead of 256. I will chage this parameter in my next tests.

Regards,

Tom.

[velmarin]

A small program, 36ks,
Hash calculation, formulates Standard and Houdini.
activating and deactivate larges pages.

Courtesy of Hall2010

http://www.mediafire.com/?wbk1k800cjvv3br

[kranium]

Hi all,

Thank you for the info, very useful .

Please correct me if I am wrong, I guess the calculation is not applicable if time per move is long, e.g. 15 mins per move.

Best wishes,
Koh, Kah Huat

i believe it applies in most (realistic) situations...
why not?

of course: you know as well as I that 15 minutes per move is an extreme example...
and the physical RAM of any system is limited.
(we're talking here about typical engine vs engine matches, not about letting your system analyze a position overnight)

modern systems today have oodles of RAM...why not allocate it for for engine use?
(are you running Photoshop and editing videos at the same time?)
bad idea

PS-
many engines send a UCI info string HashFull
watch it
if it reaches 100%...my opinion: not good

Bob Hyatt can explain the 'hashfull' cost much better than I since he actually invented some/much of it...

[PaulieD]

Somebody just mailed me this posted in another forum.

Thoughts? Interesting the he concludes results dramatically different



"Hash Table Size for Engine Matches
I am changing my hash table formula's for engine matches. As time has gone by and hardware and software changes have taken place ( have gotten better and better) in today's environment it is clear that more hash can and is being accessed than by the old formula's that used to be used.
The newer rule of thumb that I will now use is simply if the hard drive is not grinding away from using too much than you are fine.
Here is a reference from Steve Lopez about this topic,it is 8 years old, so it is even more true today.
http://www.chessexchange.com/forum/v...php?f=7&t=1397

I just doubled hash in 1 minute games from 256 to 512 and the results were dramatically different. You can almost tell the engines hunger for more ram, and this is on my slow i3 dual core system. I will also increase Little Blitzer hash proportionately and watch the differences."

This was me and here are the results that helped me draw this conclusion:

i3 Dual Core @2.53 GHz - 6.0GB RAM
W7 Home Premium X.64 SP-1
GUI: Deep Fritz 13
HASH: 256 or 512
TABLEBASES: None
BOOK: 5000 8move colors reversing
PONDER: OFF
TIME: 1 minute

Code: Select all

0813 vs Kom,256H Blitz 1m  0
                                      
1   Komodo 5.1r2 64-bit          +10  +26/=51/-23 51.50%   51.5/100
2   Stockfish 130813 64 SSE4.2   -10  +23/=51/-26 48.50%   48.5/100

Code: Select all

0813 vs Kom, 512H Blitz 1m  0
                                    
1   Stockfish 130813 64 SSE4.2   +38  +33/=45/-22 55.50%   55.5/100
2   Komodo 5.1r2 64-bit          -38  +22/=45/-33 44.50%   44.5/100

The only difference in these 2 matches is that I doubled the hash in the second 100 games. We can see S3 0813 gained 48 ELO by doing that.
I since have tried the same idea in utra fast matches and the same held to be true there in 3 matches (available to be posted).

I see here that my real results were a 48 ELO difference from this change, all these theoretical recommendations posted here is what I always also followed, but I wanted some real world tests to confirm or deny their validity. At least in this case with these samples doubling the ram in what seems to be too short of a period of time to do so worked significantly well for the sample size used.

[PaulieD]

Here I tried the same idea again, the doubling of the hash until performance actually decreased. That is what happened here with 1 GB Hash:

Code: Select all

0813 vs Kom,1024H, Blitz 1m  0
                                      
1   Komodo 5.1r2 64-bit           +7  +29/=44/-27 51.00%   51.0/100
2   Stockfish 130813 64 SSE4.2    -7  +27/=44/-29 49.00%   49.0/100

So, it appears that in these 3 - 100 game tests at 1 minute TC's that 512H worked best.
The same thing happened with ultra fast:
32 was what was considered normal
64 was better
128 was best
256 produced the reduced performance , so that 128MB hash produced the best results in Little Blitzer 15-16 second matches.

All in all 900 games were played in 6 different matches to come to this conclusion. 300 games (3x100) were at a 1 minute time control and 600 games (3x200) were played at 15-16 sec time control.