Syzygy on RAM Drive

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9757
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Syzygy on RAM Drive

Post by Laskos » Fri May 23, 2014 8:19 pm

On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).

Code: Select all

 1.  SF Syzygy RAM            	530.5/1000	148-87-765
 2.  SF NO Syzygy             	469.5/1000	87-148-765

Code: Select all

 1.  K7a Syzygy RAM           	514.0/1000	121-93-786
 2.  K7a NO Syzygy            	486.0/1000	93-121-786

Code: Select all

 1.  H4 Syzygy RAM            	520.0/1000	142-102-756
 2.  H4 NO Syzygy             	480.0/1000	102-142-756
Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.

User avatar
Laskos
Posts: 9757
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Syzygy on RAM Drive

Post by Laskos » Sun May 25, 2014 10:03 am

Laskos wrote:On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).

Code: Select all

 1.  SF Syzygy RAM            	530.5/1000	148-87-765
 2.  SF NO Syzygy             	469.5/1000	87-148-765

Code: Select all

 1.  K7a Syzygy RAM           	514.0/1000	121-93-786
 2.  K7a NO Syzygy            	486.0/1000	93-121-786

Code: Select all

 1.  H4 Syzygy RAM            	520.0/1000	142-102-756
 2.  H4 NO Syzygy             	480.0/1000	102-142-756
Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.
1GB RAMDisk with the 5-men Syzygy, positions are 6 men (4+2 and 3+3) _hard_ TB wins. I get clearer and more significant statistically results. Games 1' + 1'' on one core:

Code: Select all

 1.  SF Syzygy RAM            	608.5/1000	426-209-365
 2.  SF NO Syzygy             	391.5/1000	209-426-365

Code: Select all

 1.  K7a Syzygy RAM           	599.0/1000	431-233-336
 2.  K7a NO Syzygy            	401.0/1000	233-431-336

Code: Select all

 1.  H4 Syzygy RAM            	568.5/1000	339-202-459
 2.  H4 NO Syzygy             	431.5/1000	202-339-459
On an 8GB RAMDisk I put the 5-men Nalimovs, and I get similar improvement of H4 as with Syzygy:

Code: Select all

Nalimov Cache 128MB
 1.  H4 Nalimov RAM              569.0/1000	379-241-380
 2.  H4 NO TB                 	431.0/1000	241-379-380
To note that on these 6 men hard win positions, 5-men on RAMDisk can improve the play by some 70 Elo points.

User avatar
AdminX
Posts: 5292
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Syzygy on RAM Drive

Post by AdminX » Sun May 25, 2014 10:16 am

Hello Kai,

What OS and software did you use to create the RAMDisk?
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

User avatar
Laskos
Posts: 9757
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Syzygy on RAM Drive

Post by Laskos » Sun May 25, 2014 10:24 am

AdminX wrote:Hello Kai,

What OS and software did you use to create the RAMDisk?
Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.

User avatar
AdminX
Posts: 5292
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Syzygy on RAM Drive

Post by AdminX » Sun May 25, 2014 10:29 am

Laskos wrote:
AdminX wrote:Hello Kai,

What OS and software did you use to create the RAMDisk?
Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.
Thanks Kai,

I am going to test out ImDisk

http://www.tekrevue.com/tip/create-10-g ... k-windows/

http://reboot.pro/files/download/284-imdisk-toolkit/
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

syzygy
Posts: 4460
Joined: Tue Feb 28, 2012 10:56 pm

Re: Syzygy on RAM Drive

Post by syzygy » Sun May 25, 2014 2:18 pm

In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.

BBauer
Posts: 658
Joined: Wed Mar 08, 2006 7:58 pm

Re: Syzygy on RAM Drive

Post by BBauer » Sun May 25, 2014 2:49 pm

While you may be right in general it will not hurt too much if you use a 480 MB RAM Disk for 5-piece WDL. Especially if you have a slow hard disk.
When using 6-piece WDL and little memory bases may become swaped out and have to be reloaded. That may cost time.
However, if you have the bases on SSD and enough memory the situation may be different.

Kind regards
Bernhard

User avatar
Laskos
Posts: 9757
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Syzygy on RAM Drive

Post by Laskos » Sun May 25, 2014 4:11 pm

syzygy wrote:In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.
With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions. About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?
By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.

shrapnel
Posts: 1261
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Syzygy on RAM Drive

Post by shrapnel » Sun May 25, 2014 4:44 pm

RAM disks not worth it...SSDs are a much better option.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

syzygy
Posts: 4460
Joined: Tue Feb 28, 2012 10:56 pm

Re: Syzygy on RAM Drive

Post by syzygy » Sun May 25, 2014 6:21 pm

Laskos wrote:
syzygy wrote:In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.
With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions.
That's why I started with "in general". In your specific case there is no problem and it may help to some extent to get a more stable testing environment.

Btw, on Linux the 5-piece tables can be preloaded into RAM with a simple:

Code: Select all

$ cat /path/to/tbs/??????.rtbw >/dev/null
or if vmtouch is installed:

Code: Select all

$ vmtouch -t /path/to/tbs/??????.rtbw
The vmtouch command has other useful applications, for example to find out how much of each table is cached in RAM:

Code: Select all

$ vmtouch -m 10G -v /path/to/tbs/*.rtbw
About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?
I'm not in any way in competition with Eugene Nalimov. The concept of "fairness" seems to be foreign to the present topic.
By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.
If you have (literally) more than enough RAM then sure, but in general no. Nalimov files are also cached in the page cache (and then separately decompressed in the engine's TB cache). Loading them on RAM disk will help during the first accesses (it will not take much time to copy some relevant parts from RAM disk into the page cache) but not in the long run (RAM available for the page cache will be much smaller).

Post Reply