How Much RAM Is To Much RAM ??

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Edmund
Posts: 668
Joined: Mon Dec 03, 2007 2:01 pm
Location: Barcelona, Spain
Contact:

Re: How Much RAM Is To Much RAM ??

Post by Edmund » Sat Jan 26, 2008 9:39 am

I don't know exactly about Shredder, but in general a chess program can be written to take advantage of any amount of RAM. Just for these high values they should be specially optimised.
Lets say you had no optimisation and you wanted to look for a hash value in 128GB ram. You had to run through all entries to find out if any information is stored on a specific position. That would take too much time. But if optimised, you would have the avialable ram devided into seperate blocks to speed up the lookup.

The question is as well, if this amount is useful. In my opinion, you would only need this amount for very large timecontrols.

Another little Idea. HashLookup could be very much speeded up this way. What if you used the Hashkey (4Byte) - 4,3 * 10^9 and with it directly indexed a position in the RAM.
128 GB = 128 * 1024^3 = 1,4 * 10^11 Byte

You would have exactly 32 Byte per position to store information
(1,4 * 10^11 / (4,3 * 10^9) )

The positive effect would be, that no time would be wasted scanning through the hash tables anymore. The question of replacement falls away. So I think 128 GB of RAM could be used very much to the benefit of the chess program.

//4 Byte would probably not be enough to uniquely index a position. So I would use some of the 32Byte to verify the position.

And sorry again for not answering your first post Michael O'Brian, its just that I dont know enough about the way Shredder uses RAM for hashtables. You better ask someone from their programming-team. For others it would probably just be speculation.

Chess Gator

Re: How Much RAM Is To Much RAM ??

Post by Chess Gator » Sat Jan 26, 2008 11:11 am

tano-urayoan wrote:Original post:
Ok are you interested in 128 megs or 128 gigs?
Gigs

User avatar
Murat
Posts: 160
Joined: Thu Mar 09, 2006 4:57 am
Location: Canada

Re: How Much RAM Is To Much RAM ??

Post by Murat » Sat Jan 26, 2008 1:12 pm

On a 64 bit operating system, a 64bit engine should make use of that amount of ram

In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
May be useful in overnight or longer analyses though. Also cashing the end game databases in to extra ram will improve performance in same cases.

Chess Gator

Re: How Much RAM Is To Much RAM ??

Post by Chess Gator » Sat Jan 26, 2008 2:52 pm

Murat wrote:In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
Hi, can you explain to me what this means and why it would occur, thank you.

Edmund
Posts: 668
Joined: Mon Dec 03, 2007 2:01 pm
Location: Barcelona, Spain
Contact:

Re: How Much RAM Is To Much RAM ??

Post by Edmund » Sat Jan 26, 2008 3:02 pm

Chess Gator wrote:
Murat wrote:In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
Hi, can you explain to me what this means and why it would occur, thank you.

diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.

User avatar
Ovyron
Posts: 4160
Joined: Tue Jul 03, 2007 2:30 am

Re: How Much RAM Is To Much RAM ??

Post by Ovyron » Sat Jan 26, 2008 5:57 pm

Codeman wrote:diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.
I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.
Your beliefs create your reality, so be careful what you wish for.

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: How Much RAM Is To Much RAM ??

Post by smirobth » Sat Jan 26, 2008 10:09 pm

Ovyron wrote:
Codeman wrote:diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.
I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.
I pretty much agree. I did an experiment once, seeing how quickly chess programs could solve test positions with various amounts of RAM for hash tables. The diminishing returns kicks in pretty quickly. Going from one MB to two made a fairly dramatic difference, but if I recall my results correctly once I got to around 64MB or so I didn't see much of an improvement in solution times. In fact rather oddly there were some times when adding more RAM increased the solution time. I'm not sure why that was (perhaps the search tree was affected by the hash table size?), but it made determining exactly how much increased hash table sizes improve performance difficult to figure out exactly.

Also an exception to the diminishing returns above 64MB or so might be endgames, since my test positions didn't include those, and there are a greater number of transposition possibilities in the endgame. And of course for super long time controls the extra RAM would help more than at fast time controls. But most of the time going above 2GB will be a complete waste.
- Robin Smith

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: How Much RAM Is To Much RAM ??

Post by smirobth » Sat Jan 26, 2008 10:23 pm

Chess Gator wrote:
Murat wrote:In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
Hi, can you explain to me what this means and why it would occur, thank you.
Others have explained what diminishing returns are. Why it happens is because hash tables only help when the search reaches a position that has already been evaluated previously (arrived at via a different move order). But there are a limited number of reasonable move sequences that will lead a program back to a previously evaluated position. Thus the larger the hash table, the lower the percentage of entries in the table that result in hash table "hits".
- Robin Smith

Chess Gator

Re: How Much RAM Is To Much RAM ??

Post by Chess Gator » Sat Jan 26, 2008 10:24 pm

Ovyron wrote:
Codeman wrote:diminishing returns mean in this case, that the advantages get smaller, the more ram you add.
so the first GB ram you add brings a higher benefit, than the second GB, or the third.
I think that it gets so diminishing that adding above 2 GB of RAM will add less than 1 ELO in strength.
WOW :shock: that is amazing, I never would have guessed only 2GB is required for max effect! I wonder why?

If increased RAM will not make a significant difference, what would be the most important piece of hardware to upgrade, would u say :?:

User avatar
smirobth
Posts: 2307
Joined: Wed Mar 08, 2006 7:41 pm
Location: Brownsville Texas USA

Re: How Much RAM Is To Much RAM ??

Post by smirobth » Sat Jan 26, 2008 10:36 pm

smirobth wrote:
Chess Gator wrote:
Murat wrote:In my opinion with current hardware, that much ram will return extremely diminishing returns in tournament modes.
Hi, can you explain to me what this means and why it would occur, thank you.
Others have explained what diminishing returns are. Why it happens is because hash tables only help when the search reaches a position that has already been evaluated previously (arrived at via a different move order). But there are a limited number of reasonable move sequences that will lead a program back to a previously evaluated position. Thus the larger the hash table, the lower the percentage of entries in the table that result in hash table "hits".
I should mention that there is another cause of diminishing returns, which is that it takes a while for a hash table to even become fully populated with positions. If you had a program that searched 1 millions nodes per second, and used a 128GB hash table, it would take 35 hours before the hash table was even filled with entries! Even a 2GB table would take half an hour to be completely filled.
- Robin Smith

Post Reply