7-men Syzygy attempt

Discussion of chess software programming and technical issues.

Moderators: hgm, chrisw, Rebel

noobpwnftw
Posts: 694
Joined: Sun Nov 08, 2015 11:10 pm
Full name: Bojun Guo

Re: 7-men Syzygy attempt

Post by noobpwnftw »

Dann Corbit wrote: I will have to buy more RAM then. My biggest machines only have 128 GB RAM installed.
Either that, or it may be possible to use more space efficient indexing scheme during generation, trading some speed for less memory usage, that may happen in the future.
niklasf
Posts: 42
Joined: Sat May 16, 2015 11:41 pm

Re: 7-men Syzygy attempt

Post by niklasf »

Awesome. Would you mind creating checksums of those files? I'll also check the file times, but checksums should be more robust and exclude other potential errors.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: 7-men Syzygy attempt

Post by duncan »

Dann Corbit wrote:
Where do I find the modified code.
I might give a try at making a full set of 7 man files.
Also, I would like to get the statistics files for the 6v1 files if this is possible.
how long do you estimate it should take ?
Dann Corbit
Posts: 12699
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: 7-men Syzygy attempt

Post by Dann Corbit »

duncan wrote:
Dann Corbit wrote:
Where do I find the modified code.
I might give a try at making a full set of 7 man files.
Also, I would like to get the statistics files for the 6v1 files if this is possible.
how long do you estimate it should take ?
I think it is very hard to guess.
I would probably use a 64 core machine with 1.2 TB RAM and very fast SAS storage.

If I were to buy all the storage right now, it would be quite expensive.
So I will probably buy as space is needed.

Here is a machine that could be used as a minimum configuration:
https://www.ebay.com/itm/HP-ProLiant-DL ... Sw6WdXip14
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: 7-men Syzygy attempt

Post by duncan »

Dann Corbit wrote:
duncan wrote:
Dann Corbit wrote:
Where do I find the modified code.
I might give a try at making a full set of 7 man files.
Also, I would like to get the statistics files for the 6v1 files if this is possible.
how long do you estimate it should take ?
I think it is very hard to guess.
I would probably use a 64 core machine with 1.2 TB RAM and very fast SAS storage.

If I were to buy all the storage right now, it would be quite expensive.
So I will probably buy as space is needed.

Here is a machine that could be used as a minimum configuration:
https://www.ebay.com/itm/HP-ProLiant-DL ... Sw6WdXip14
are the sas drives the major expense ?
syzygy
Posts: 5654
Joined: Tue Feb 28, 2012 11:56 pm

Re: 7-men Syzygy attempt

Post by syzygy »

A couple of regular HDDs suffice for generating the 7-men TBs. The expensive part is the amount of RAM needed and the system that supports such an amount.

For accessing the TBs at full speed during analysis, a 10 TB SSD would be nice to have. For generation you can do without an SSD since only a limited number of TBs are accessed at any one time.
noobpwnftw
Posts: 694
Joined: Sun Nov 08, 2015 11:10 pm
Full name: Bojun Guo

Re: 7-men Syzygy attempt

Post by noobpwnftw »

Dann Corbit wrote: Here is a machine that could be used as a minimum configuration:
https://www.ebay.com/itm/HP-ProLiant-DL ... Sw6WdXip14
That does look like a deal, however, mine had serious memory bus congestion, not sure how would it go with less than a half of memory clock, you might want a test run if that's possible.

For a test run, you can compile the generator from refactor branch, prepare a small package of all required subsidiary tables of a 7-piece set and have the seller run it for you.

Quick grab of some random time stats during generation for your reference:

Code: Select all

time taken =   0:29.080
Calculating white captures.
time taken =   0:09.456
time taken =   2:22.732
Calculating black captures.
time taken =   0:08.343
time taken =   0:48.656
time taken =   0:47.865
time taken =   0:42.034
time taken =   0:39.249
Calculating mate positions.
time taken =   0:33.390
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: 7-men Syzygy attempt

Post by duncan »

Dann Corbit wrote:
duncan wrote:
Dann Corbit wrote:
Where do I find the modified code.
I might give a try at making a full set of 7 man files.
Also, I would like to get the statistics files for the 6v1 files if this is possible.
how long do you estimate it should take ?
I think it is very hard to guess.
I would probably use a 64 core machine with 1.2 TB RAM and very fast SAS storage.

If I were to buy all the storage right now, it would be quite expensive.
So I will probably buy as space is needed.

Here is a machine that could be used as a minimum configuration:
https://www.ebay.com/itm/HP-ProLiant-DL ... Sw6WdXip14
any possibilty of working together with Bojun Guo to speed up the time it takes?
Dann Corbit
Posts: 12699
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: 7-men Syzygy attempt

Post by Dann Corbit »

duncan wrote:
Dann Corbit wrote:
duncan wrote:
Dann Corbit wrote:
Where do I find the modified code.
I might give a try at making a full set of 7 man files.
Also, I would like to get the statistics files for the 6v1 files if this is possible.
how long do you estimate it should take ?
I think it is very hard to guess.
I would probably use a 64 core machine with 1.2 TB RAM and very fast SAS storage.

If I were to buy all the storage right now, it would be quite expensive.
So I will probably buy as space is needed.

Here is a machine that could be used as a minimum configuration:
https://www.ebay.com/itm/HP-ProLiant-DL ... Sw6WdXip14
any possibilty of working together with Bojun Guo to speed up the time it takes?
At some point I will probably join the effort.

For sure, I would download his completed files as that would accelerate any new efforts.

For now, I am just looking at a strategy. I might wait as long as a year before I start purchasing the needed equipment.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
niklasf
Posts: 42
Joined: Sat May 16, 2015 11:41 pm

Re: 7-men Syzygy attempt

Post by niklasf »

This position shows an error in either the probing code, my Python port of it or the KRBNvKQN table:

[d]QN4n1/3b4/3k4/8/6r1/2K5/8/8 w - -

It is given as winning for white with DTZ 11: https://syzygy-tables.info/?fen=QN4n1/3 ... _w_-_-_0_1

But for example the position after Qa6+ is given as -4, which is too large of a jump.