can you have to much hash?

[adams161]

I went from testing on 128 megs of hash to 256 megs of hash. At first last night it seemed like average nps might be somewhat higher. Any search call is a node even if it returns a hash value so more hash cuttofs could make it seem faster. Now its been on awhile and it seems nps are somewhat lower. the higher and lower bounds are not huge maybe +-10%, and my testing of eyeballing nps over various games is not perfect.

Im wondering if you can have to big a hash table. In positions were hash really comes into play and it finds a lot of good hash hits I suppose more hash wins. but on average perhaps the cost of the bigger mod operation , i do modnum=hnumber % hashmax, can hurt you if you are to large a hash table for the extra hash to benefit you beyond the slower operational speed of running bigger hash.

There may be no one definite answer on this and could depend on my comptuers specs i suppose but just throwing out the question. Rating wise its playing lower but i think it was close to a best last night when i started the higher hash, also people have been playing more 5 0 today and more 3 0 yesterday and i think it performs better at 3 0. but I intentionally allow both figuring the 5 0 anchors it a bit.

In looking at its games i dont see anything particularly broken, its play seems sharp enough, just wondering if i have slowed it down a bit.

Mike

[wgarvin]

I went from testing on 128 megs of hash to 256 megs of hash. At first last night it seemed like average nps might be somewhat higher. Any search call is a node even if it returns a hash value so more hash cuttofs could make it seem faster. Now its been on awhile and it seems nps are somewhat lower. the higher and lower bounds are not huge maybe +-10%, and my testing of eyeballing nps over various games is not perfect.

Im wondering if you can have to big a hash table. In positions were hash really comes into play and it finds a lot of good hash hits I suppose more hash wins. but on average perhaps the cost of the bigger mod operation , i do modnum=hnumber % hashmax, can hurt you if you are to large a hash table for the extra hash to benefit you beyond the slower operational speed of running bigger hash.

As long as hashmax is a power of 2, the mod operation will make no difference.

The speed difference is more likely to be something to do with the memory hierarchy (TLB misses, page faults, etc.)

[adams161]

I should add that my computers specs are pentium 4, 2.19 gigahertz, and i upgraded ram from 512 megs last night to 1 gigabyte. Its a dell. I was thinking pottentially there could be in addition to the mod operator having to be bigger, memory access speeds on this type of computer as you grow hash in ram.

Mike

[adams161]

i define hashmax as 1024 and hashbase as 512.

I multiply hashmax * hashbase for my base hash.

then i apply a multiplier the user ( including me) can set at run time. normal hash is a multiplier of 8 with no intervention. I'm using a multiplier of 16 now.

so it was 1024*512*8 now its 1024 * 512* 16. That should be a power of two. hashmax is stored in an unsigned long since int is to small with my compiler i think.

so you have:
unsigned long modnum
unsigned long hashmax
and _int64 hnumber and i do

modnum = hnumber%hashmax.

if my choice of variables are fine, i am working with powers of 2. so it must be eathier, a ghost thing and i'm not really noticing it slower just circ umstances are different. or something with my computer.

Mike

[adams161]

the _int64 is unsigned to

[bob]

I went from testing on 128 megs of hash to 256 megs of hash. At first last night it seemed like average nps might be somewhat higher. Any search call is a node even if it returns a hash value so more hash cuttofs could make it seem faster. Now its been on awhile and it seems nps are somewhat lower. the higher and lower bounds are not huge maybe +-10%, and my testing of eyeballing nps over various games is not perfect.

Im wondering if you can have to big a hash table. In positions were hash really comes into play and it finds a lot of good hash hits I suppose more hash wins. but on average perhaps the cost of the bigger mod operation , i do modnum=hnumber % hashmax, can hurt you if you are to large a hash table for the extra hash to benefit you beyond the slower operational speed of running bigger hash.

There may be no one definite answer on this and could depend on my comptuers specs i suppose but just throwing out the question. Rating wise its playing lower but i think it was close to a best last night when i started the higher hash, also people have been playing more 5 0 today and more 3 0 yesterday and i think it performs better at 3 0. but I intentionally allow both figuring the 5 0 anchors it a bit.

In looking at its games i dont see anything particularly broken, its play seems sharp enough, just wondering if i have slowed it down a bit.

Mike

If there were no access penalty, bigger hash is _never_ harmful. However, there is a penalty. Bigger hash means more pages of memory, which means the TLB becomes the bottleneck, particularly on 64 bit machines where a TLB miss is monstrously expensive.

[sje]

If there were no access penalty, bigger hash is _never_ harmful. However, there is a penalty. Bigger hash means more pages of memory, which means the TLB becomes the bottleneck, particularly on 64 bit machines where a TLB miss is monstrously expensive.

Well, there is the possibility that the page size could be variable depending on the hardware, the OS, and the amount of memory. Also, interference from any background processes could affect engine memory access times differently depending on how much memory is left for them or in the order in which they were launched.

Another factor is the physical memory configuration in a multiple DIMM box. If not all the DIMMs are the same capacity or some are unpaired, then memory access time could vary depending on the size and location of the transposition table.

So not all cache misses to main memory are the same. But they all will be fairly close time wise.

[bob]

If there were no access penalty, bigger hash is _never_ harmful. However, there is a penalty. Bigger hash means more pages of memory, which means the TLB becomes the bottleneck, particularly on 64 bit machines where a TLB miss is monstrously expensive.

Well, there is the possibility that the page size could be variable depending on the hardware, the OS, and the amount of memory. Also, interference from any background processes could affect engine memory access times differently depending on how much memory is left for them or in the order in which they were launched.

I don't have background processes when I play chess. And intel has two page sizes that the O/S can choose. But that is really irrelevant, as it just changes the max size by 1024 or so, so as I said, the TLB is the issue that is going to make or break a bigger hash.

Another factor is the physical memory configuration in a multiple DIMM box. If not all the DIMMs are the same capacity or some are unpaired, then memory access time could vary depending on the size and location of the transposition table.

Never run on such a box, and anyone using one for chess is an idiot.

So not all cache misses to main memory are the same. But they all will be fairly close time wise.

Not at all. You can get a cache miss after a TLB hit and it is just one memory latency away. A TLB miss can add 4 latency periods on top of that on a 64 bit box since the page table is 4 levels deep. 64 bit boxes can see a 5x differential in access times depending on whether you have a cache that is too small, or a TLB that is too small. Both would be a real killer of course.

[adams161]

sje wrote:
Another factor is the physical memory configuration in a multiple DIMM box. If not all the DIMMs are the same capacity or some are unpaired, then memory access time could vary depending on the size and location of the transposition table.

bob wrote:
Never run on such a box, and anyone using one for chess is an idiot.

I wanted to understand this. the dimm is the ram chip? i had one 512 meg before now i have two 512 megs filling both slots. i think the computer was suppose to take a 2300 or 2100 speed ram chip but the store didnt have that but they had a 3200 or 3300 and it said on teh back it was downward compatible with the lower speeds. Is this an issue? it sounds like i'm getting nothing out of this maybe i shoudl pull the ram chip out.

Mike

[Bo Persson]

sje wrote:
Another factor is the physical memory configuration in a multiple DIMM box. If not all the DIMMs are the same capacity or some are unpaired, then memory access time could vary depending on the size and location of the transposition table.

bob wrote:
Never run on such a box, and anyone using one for chess is an idiot.

I wanted to understand this. the dimm is the ram chip? i had one 512 meg before now i have two 512 megs filling both slots. i think the computer was suppose to take a 2300 or 2100 speed ram chip but the store didnt have that but they had a 3200 or 3300 and it said on teh back it was downward compatible with the lower speeds. Is this an issue? it sounds like i'm getting nothing out of this maybe i shoudl pull the ram chip out.

Mike

Not likely. Some newer CPUs have dual or even tripple channel memory access. In those cases, you shold always use two or three identical chips for each memory bank.

If you had a single DIMM before, adding another one shouldn't have this effect *unless* the machine totally misidentifies the new one and lowers the memory speed. Very unlikely.

You could change places of the two DIMMs and see if that makes any difference.