Page 1 of 3
Syzygy on RAM Drive
Posted: Fri May 23, 2014 10:19 pm
by Laskos
On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).
Code: Select all
1. SF Syzygy RAM 530.5/1000 148-87-765
2. SF NO Syzygy 469.5/1000 87-148-765
Code: Select all
1. K7a Syzygy RAM 514.0/1000 121-93-786
2. K7a NO Syzygy 486.0/1000 93-121-786
Code: Select all
1. H4 Syzygy RAM 520.0/1000 142-102-756
2. H4 NO Syzygy 480.0/1000 102-142-756
Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 12:03 pm
by Laskos
Laskos wrote:On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).
Code: Select all
1. SF Syzygy RAM 530.5/1000 148-87-765
2. SF NO Syzygy 469.5/1000 87-148-765
Code: Select all
1. K7a Syzygy RAM 514.0/1000 121-93-786
2. K7a NO Syzygy 486.0/1000 93-121-786
Code: Select all
1. H4 Syzygy RAM 520.0/1000 142-102-756
2. H4 NO Syzygy 480.0/1000 102-142-756
Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.
1GB RAMDisk with the 5-men Syzygy, positions are 6 men (4+2 and 3+3) _hard_ TB wins. I get clearer and more significant statistically results. Games 1' + 1'' on one core:
Code: Select all
1. SF Syzygy RAM 608.5/1000 426-209-365
2. SF NO Syzygy 391.5/1000 209-426-365
Code: Select all
1. K7a Syzygy RAM 599.0/1000 431-233-336
2. K7a NO Syzygy 401.0/1000 233-431-336
Code: Select all
1. H4 Syzygy RAM 568.5/1000 339-202-459
2. H4 NO Syzygy 431.5/1000 202-339-459
On an 8GB RAMDisk I put the 5-men Nalimovs, and I get similar improvement of H4 as with Syzygy:
Code: Select all
Nalimov Cache 128MB
1. H4 Nalimov RAM 569.0/1000 379-241-380
2. H4 NO TB 431.0/1000 241-379-380
To note that on these 6 men hard win positions, 5-men on RAMDisk can improve the play by some 70 Elo points.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 12:16 pm
by AdminX
Hello Kai,
What OS and software did you use to create the RAMDisk?
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 12:24 pm
by Laskos
AdminX wrote:Hello Kai,
What OS and software did you use to create the RAMDisk?
Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 12:29 pm
by AdminX
Laskos wrote:AdminX wrote:Hello Kai,
What OS and software did you use to create the RAMDisk?
Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.
Thanks Kai,
I am going to test out ImDisk
http://www.tekrevue.com/tip/create-10-g ... k-windows/
http://reboot.pro/files/download/284-imdisk-toolkit/
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 4:18 pm
by syzygy
In general I can only advise against the use of RAM disks for storing TB files.
If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.
If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.
A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 4:49 pm
by BBauer
While you may be right in general it will not hurt too much if you use a 480 MB RAM Disk for 5-piece WDL. Especially if you have a slow hard disk.
When using 6-piece WDL and little memory bases may become swaped out and have to be reloaded. That may cost time.
However, if you have the bases on SSD and enough memory the situation may be different.
Kind regards
Bernhard
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 6:11 pm
by Laskos
syzygy wrote:In general I can only advise against the use of RAM disks for storing TB files.
If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.
If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.
A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.
With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions. About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?
By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 6:44 pm
by shrapnel
RAM disks not worth it...SSDs are a much better option.
Re: Syzygy on RAM Drive
Posted: Sun May 25, 2014 8:21 pm
by syzygy
Laskos wrote:syzygy wrote:In general I can only advise against the use of RAM disks for storing TB files.
If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.
If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.
A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.
With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions.
That's why I started with "in general". In your specific case there is no problem and it may help to some extent to get a more stable testing environment.
Btw, on Linux the 5-piece tables can be preloaded into RAM with a simple:
Code: Select all
$ cat /path/to/tbs/??????.rtbw >/dev/null
or if vmtouch is installed:
Code: Select all
$ vmtouch -t /path/to/tbs/??????.rtbw
The vmtouch command has other useful applications, for example to find out how much of each table is cached in RAM:
Code: Select all
$ vmtouch -m 10G -v /path/to/tbs/*.rtbw
About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?
I'm not in any way in competition with Eugene Nalimov. The concept of "fairness" seems to be foreign to the present topic.
By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.
If you have (literally) more than enough RAM then sure, but in general no. Nalimov files are also cached in the page cache (and then separately decompressed in the engine's TB cache). Loading them on RAM disk will help during the first accesses (it will not take much time to copy some relevant parts from RAM disk into the page cache) but not in the long run (RAM available for the page cache will be much smaller).