How Much RAM Is To Much RAM ??

[Edmund]

I don't know exactly about Shredder, but in general a chess program can be written to take advantage of any amount of RAM. Just for these high values they should be specially optimised.
Lets say you had no optimisation and you wanted to look for a hash value in 128GB ram. You had to run through all entries to find out if any information is stored on a specific position. That would take too much time. But if optimised, you would have the avialable ram devided into seperate blocks to speed up the lookup.

The question is as well, if this amount is useful. In my opinion, you would only need this amount for very large timecontrols.

Another little Idea. HashLookup could be very much speeded up this way. What if you used the Hashkey (4Byte) - 4,3 * 10^9 and with it directly indexed a position in the RAM.
128 GB = 128 * 1024^3 = 1,4 * 10^11 Byte

You would have exactly 32 Byte per position to store information
(1,4 * 10^11 / (4,3 * 10^9) )

The positive effect would be, that no time would be wasted scanning through the hash tables anymore. The question of replacement falls away. So I think 128 GB of RAM could be used very much to the benefit of the chess program.

//4 Byte would probably not be enough to uniquely index a position. So I would use some of the 32Byte to verify the position.

And sorry again for not answering your first post Michael O'Brian, its just that I dont know enough about the way Shredder uses RAM for hashtables. You better ask someone from their programming-team. For others it would probably just be speculation.

[Chess Gator]

Original post:
Ok are you interested in 128 megs or 128 gigs?

Gigs

[Murat]

On a 64 bit operating system, a 64bit engine should make use of that amount of ram

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
May be useful in overnight or longer analyses though. Also cashing the end game databases in to extra ram will improve performance in same cases.

[Chess Gator]

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.

Hi, can you explain to me what this means and why it would occur, thank you.

[Edmund]

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.

Hi, can you explain to me what this means and why it would occur, thank you.

diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.

[Ovyron]

diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.

I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.

[smirobth]

diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.

I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.

I pretty much agree. I did an experiment once, seeing how quickly chess programs could solve test positions with various amounts of RAM for hash tables. The diminishing returns kicks in pretty quickly. Going from one MB to two made a fairly dramatic difference, but if I recall my results correctly once I got to around 64MB or so I didn't see much of an improvement in solution times. In fact rather oddly there were some times when adding more RAM increased the solution time. I'm not sure why that was (perhaps the search tree was affected by the hash table size?), but it made determining exactly how much increased hash table sizes improve performance difficult to figure out exactly.

Also an exception to the diminishing returns above 64MB or so might be endgames, since my test positions didn't include those, and there are a greater number of transposition possibilities in the endgame. And of course for super long time controls the extra RAM would help more than at fast time controls. But most of the time going above 2GB will be a complete waste.

[smirobth]

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.

Hi, can you explain to me what this means and why it would occur, thank you.

Others have explained what diminishing returns are. Why it happens is because hash tables only help when the search reaches a position that has already been evaluated previously (arrived at via a different move order). But there are a limited number of reasonable move sequences that will lead a program back to a previously evaluated position. Thus the larger the hash table, the lower the percentage of entries in the table that result in hash table "hits".

[Chess Gator]

diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.

I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.

WOW that is amazing, I never would have guessed only 2GB is required for max effect! I wonder why?

If increased RAM will not make a significant difference, what would be the most important piece of hardware to upgrade, would u say

[smirobth]

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.

Hi, can you explain to me what this means and why it would occur, thank you.

Others have explained what diminishing returns are. Why it happens is because hash tables only help when the search reaches a position that has already been evaluated previously (arrived at via a different move order). But there are a limited number of reasonable move sequences that will lead a program back to a previously evaluated position. Thus the larger the hash table, the lower the percentage of entries in the table that result in hash table "hits".

I should mention that there is another cause of diminishing returns, which is that it takes a while for a hash table to even become fully populated with positions. If you had a program that searched 1 millions nodes per second, and used a 128GB hash table, it would take 35 hours before the hash table was even filled with entries! Even a 2GB table would take half an hour to be completely filled.