Syzygy 7 piece Machine slow down

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
first25plus5
Posts: 10
Joined: Sat Jul 22, 2017 12:50 am
Location: New Zealand

Syzygy 7 piece Machine slow down

Post by first25plus5 » Wed Aug 28, 2019 7:51 pm

Hi my computer system basics are as follows: Windows 10 Pro, 3.1Ghz, 16BG RAM, 64bit-Quad core. I have the more important 7piece Table bases downloaded onto a SSD SATA drive totalling 443GB and more on another drive totalling 219GB. While I'm running Stockfish10 it isn't very long before (going by task manager) the CPU almost suddenly grinds to a virtual halt and also the C-drive usage increases dramatically. I have varied the hashtable settings and the Virtual Memory in the system, as there is enough space on the SSD drives to be inside the computer system's own recommendations on virtual memory settings. I suspect perhaps the 7-piece syzygy files are so big that I need tons more RAM. Does anyone have any suggestions please. Thanks in advance.

first25plus5
Posts: 10
Joined: Sat Jul 22, 2017 12:50 am
Location: New Zealand

Re: Syzygy 7 piece Machine slow down

Post by first25plus5 » Wed Aug 28, 2019 7:53 pm

Forgot to mention that I have this issue in both Arena and Chessbase, so suspect it's a memory allocation issue.

Sesse
Posts: 180
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: Syzygy 7 piece Machine slow down

Post by Sesse » Wed Aug 28, 2019 8:08 pm

More RAM would help, and most likely, your SSD is also pretty slow. Failing that, you can ask to probe less often (increase SyzygyProbeDepth).

Dann Corbit
Posts: 9895
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Syzygy 7 piece Machine slow down

Post by Dann Corbit » Wed Aug 28, 2019 8:08 pm

16 GB RAM is really small.
I guess that you are getting page faults.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

first25plus5
Posts: 10
Joined: Sat Jul 22, 2017 12:50 am
Location: New Zealand

Re: Syzygy 7 piece Machine slow down

Post by first25plus5 » Thu Aug 29, 2019 7:52 pm

Hi all thankyou very much. Yes it does appear that the main issue is far too little RAM. I've learnt a few things. Many thanks for all the help. Regards.

Modern Times
Posts: 2366
Joined: Thu Jun 07, 2012 9:02 pm

Re: Syzygy 7 piece Machine slow down

Post by Modern Times » Thu Aug 29, 2019 8:06 pm

Yes, with 16GB RAM even with 6-men my machine would start paging on long analysis. I restrict myself to 5-men now.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

syzygy
Posts: 4447
Joined: Tue Feb 28, 2012 10:56 pm

Re: Syzygy 7 piece Machine slow down

Post by syzygy » Fri Aug 30, 2019 12:08 pm

first25plus5 wrote:
Thu Aug 29, 2019 7:52 pm
Hi all thankyou very much. Yes it does appear that the main issue is far too little RAM. I've learnt a few things. Many thanks for all the help. Regards.
In addition to increasing SyzygyProbeDepth, you could try disabling the Windows pagefile (which I suppose is on the non-SSD C drive).

Most likely it will still be slow but it should stop the C-drive usage.

Moving the pagefile to the SSD would NOT be a good idea in this situation, since extreme paging is not healthy for an SSD.

I think 16GB RAM is really the lower limit for using 6-piece TBs. For 7-piece TBs, think 1TB RAM.

first25plus5
Posts: 10
Joined: Sat Jul 22, 2017 12:50 am
Location: New Zealand

Re: Syzygy 7 piece Machine slow down

Post by first25plus5 » Wed Sep 04, 2019 10:14 pm

Many thanks. Yes I totally agree with 16GB RAM as a lower limit for 6-piece tb. (15.8GB free on my system of the 16GB)
I was even having issues with 6-piece tb prior to this.
My solution for 6-piece on my machine is to set my HASH to 9 GB, and the cache setting to 1 GB. (Leaves 8GB of the 9GB HASH for the machine to work well.) Runs beautifully, and I still get a great amount of 6-pieces on my SSD stick, which I discovered I should not fill past about 15 GB to 16 GB. I have topped my SSD stick up to reach 14.9 GB as it has turned out. Anything much past that and it causes issues. Perhaps something to do with the RAM free reading the SSD drive.

Getting back to 7-piece tb, perhaps the tb files should be no larger than the amount of RAM free on a machine.
E.g. if there are 500 GB of tb files, then RAM be a little more, say 512 GB.
Cheers

syzygy
Posts: 4447
Joined: Tue Feb 28, 2012 10:56 pm

Re: Syzygy 7 piece Machine slow down

Post by syzygy » Thu Sep 05, 2019 7:44 am

first25plus5 wrote:
Wed Sep 04, 2019 10:14 pm
I was even having issues with 6-piece tb prior to this.
My solution for 6-piece on my machine is to set my HASH to 9 GB, and the cache setting to 1 GB. (Leaves 8GB of the 9GB HASH for the machine to work well.)
I'm not sure what you mean by "cache setting". If this is about the Nalimov cache (don't know what engine you're using), then it does not influence Syzygy behaviour.
Runs beautifully, and I still get a great amount of 6-pieces on my SSD stick, which I discovered I should not fill past about 15 GB to 16 GB. I have topped my SSD stick up to reach 14.9 GB as it has turned out. Anything much past that and it causes issues. Perhaps something to do with the RAM free reading the SSD drive.
If you want to get rid of the swapping, disable the Windows pagefile. It may not be the ideal solution, but it works.
Getting back to 7-piece tb, perhaps the tb files should be no larger than the amount of RAM free on a machine.
E.g. if there are 500 GB of tb files, then RAM be a little more, say 512 GB.
No, it should work fine with bigger files if Windows would just not swap out active programs to the pagefile and instead release TB data from the system cache. The easiest way to achieve this is to disable the pagefile, so Windows can't swap out active programs in the first place.

first25plus5
Posts: 10
Joined: Sat Jul 22, 2017 12:50 am
Location: New Zealand

Re: Syzygy 7 piece Machine slow down

Post by first25plus5 » Sun Sep 08, 2019 7:49 pm

Thanks again and for your patience. I'm using Stockfish and have removed the cache setting. I tried disabling the windows paging file but the computer has problem starting up.
So I have had to leave the windows paging file back on and have increased the HASH to 14GB and re-tested.
Seems fine, but can't go larger with the total size of the tb files which remain at 14.9 GB.

Post Reply