7-men Syzygy attempt

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Sesse
Posts: 181
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: 7-men Syzygy attempt

Post by Sesse » Mon Aug 20, 2018 9:08 am

phhnguyen wrote:
Mon Aug 20, 2018 12:12 am
Now I am curious about 8 men. Perhaps they will take about 4 years and 200T to build on non-supercomputers? Doable :?:
These things tend to be exponential. In particular, you'd need a machine with a lot more RAM than 1TB.

There are certainly entities that could build 8-man tables (Google, Facebook, Amazon, several state actors, probably also many others), but I don't think any of them are interested.

duncan
Posts: 10528
Joined: Mon Jul 07, 2008 8:50 pm

Re: 7-men Syzygy attempt

Post by duncan » Mon Aug 20, 2018 12:29 pm

Sesse wrote:
These things tend to be exponential. In particular, you'd need a machine with a lot more RAM than 1TB.
I thought roughly speaking you need 60 times more hard disk and ram for each extra piece.

User avatar
jshriver
Posts: 1105
Joined: Wed Mar 08, 2006 8:41 pm
Location: Morgantown, WV, USA
Contact:

Re: 7-men Syzygy attempt

Post by jshriver » Mon Aug 20, 2018 3:05 pm

Congrats to everyone for this amazing milestone!

I would like to put these on the torrent site, however I do not have the storage capacity locally to even download them.
Anyone here with a full set who would be willing to make a torrent file, pointing to oics.olympuschess.com/tracker/announce.php and share it with me? I'll add it to the server.

I'm also thinking due to the size , it would be best to have 1 torrent per directory (5-1 pawnless, 5-1 pawnful, 3-3 etc)?
It's amazing to have this available online via the web (ty noobpwnftw!), I'd just like to offer a secondary place as a backup.

-Josh

User avatar
phhnguyen
Posts: 459
Joined: Wed Apr 21, 2010 2:58 am
Location: Australia
Full name: Nguyen Hong Pham
Contact:

Re: 7-men Syzygy attempt

Post by phhnguyen » Mon Aug 20, 2018 3:14 pm

duncan wrote:
Mon Aug 20, 2018 12:29 pm
Sesse wrote:
These things tend to be exponential. In particular, you'd need a machine with a lot more RAM than 1TB.
I thought roughly speaking you need 60 times more hard disk and ram for each extra piece.
I’m sure the factor should be much smaller than 60. Long time ago someone has already predicted that building Syzygy 7 men would take 1TB Ram and 3-5 years. Actually it took not that RAM size and only 5 months!

7 men is about 17 times larger than 6 men. Thus I guess the factor for 8 men is reasonably about 10-25. Magic may happens again ;)
https://banksiagui.com
A freeware chess GUI, based on opensource Banksia - the chess tournament manager

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 7-men Syzygy attempt

Post by Dann Corbit » Mon Aug 20, 2018 5:40 pm

phhnguyen wrote:
Mon Aug 20, 2018 12:12 am
noobpwnftw wrote:
Sun Aug 19, 2018 4:04 pm
All 7-piece tables have been built. I have also renamed the folder(removed "_testing").
Exactly 5 months!
Wow, congrats!!!!

Quick glance, I see some amazing things, compared with Lomonosov EGTB:
- so short building period (5 months on non-supercomputers vs 6 months supercomputers)
- so small size (17T vs 150T)

----
Now I am curious about 8 men. Perhaps they will take about 4 years and 200T to build on non-supercomputers? Doable :?:
I guess that the horsepower of his system is pretty close to the Lomonosov supercomputer.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 7-men Syzygy attempt

Post by Dann Corbit » Mon Aug 20, 2018 5:41 pm

noobpwnftw wrote:
Sun Aug 19, 2018 4:04 pm
All 7-piece tables have been built. I have also renamed the folder(removed "_testing").

Now the full set is at:
ftp://ftp.chessdb.cn/pub/syzygy/7men/

Final sizes:

Code: Select all

151G    ./3-6men
12T     ./7men/4v3_pawnful
2.4T    ./7men/4v3_pawnless
2.3T    ./7men/5v2_pawnful
660G    ./7men/5v2_pawnless
117G    ./7men/6v1_pawnful
87G     ./7men/6v1_pawnless
17T     ./7men

Code: Select all

First file built:
2018/03/19
Last file built:
2018/08/20
Exactly 5 months!
I hope that this effort will become very famous. It is really something incredible.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Sven
Posts: 3830
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: 7-men Syzygy attempt

Post by Sven » Mon Aug 20, 2018 7:38 pm

Dann Corbit wrote:
Mon Aug 20, 2018 5:41 pm
noobpwnftw wrote:
Sun Aug 19, 2018 4:04 pm
All 7-piece tables have been built. I have also renamed the folder(removed "_testing").

Now the full set is at:
ftp://ftp.chessdb.cn/pub/syzygy/7men/

Final sizes:

Code: Select all

151G    ./3-6men
12T     ./7men/4v3_pawnful
2.4T    ./7men/4v3_pawnless
2.3T    ./7men/5v2_pawnful
660G    ./7men/5v2_pawnless
117G    ./7men/6v1_pawnful
87G     ./7men/6v1_pawnless
17T     ./7men

Code: Select all

First file built:
2018/03/19
Last file built:
2018/08/20
Exactly 5 months!
I hope that this effort will become very famous. It is really something incredible.
+1 - and I think we should not forget to mention Ronald as the author of the generator program, who has put some serious effort into adapting his program for 7-men.
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: 7-men Syzygy attempt

Post by Dann Corbit » Mon Aug 20, 2018 8:10 pm

Sesse wrote:
Mon Aug 20, 2018 9:08 am
phhnguyen wrote:
Mon Aug 20, 2018 12:12 am
Now I am curious about 8 men. Perhaps they will take about 4 years and 200T to build on non-supercomputers? Doable :?:
These things tend to be exponential. In particular, you'd need a machine with a lot more RAM than 1TB.

There are certainly entities that could build 8-man tables (Google, Facebook, Amazon, several state actors, probably also many others), but I don't think any of them are interested.
Instead of 8 man files, I think the next logical project is the 7 man DTM with Ronald's new format.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

duncan
Posts: 10528
Joined: Mon Jul 07, 2008 8:50 pm

Re: 7-men Syzygy attempt

Post by duncan » Mon Aug 20, 2018 8:58 pm

phhnguyen wrote:
Mon Aug 20, 2018 3:14 pm
duncan wrote:
Mon Aug 20, 2018 12:29 pm
Sesse wrote:
These things tend to be exponential. In particular, you'd need a machine with a lot more RAM than 1TB.
I thought roughly speaking you need 60 times more hard disk and ram for each extra piece.
I’m sure the factor should be much smaller than 60. Long time ago someone has already predicted that building Syzygy 7 men would take 1TB Ram and 3-5 years. Actually it took not that RAM size and only 5 months!

7 men is about 17 times larger than 6 men. Thus I guess the factor for 8 men is reasonably about 10-25. Magic may happens again ;)

according to this 7 man is 113 times bigger than 6 man
https://en.chessbase.com/post/endgame-t ... rt-history

Until the advent of the compressed Nalimov Tablebases, Edwards' Tablebases were quite popular and used in several chess programs, such as Crafty, Gromit, and the commercial MChess Pro, to name a few. The only problem was that the full five-piece set alone was well over 50 GB! Furthermore, if you consider that each additional piece to a tablebase takes up about 160 times more space, then a full six-piece set could be expected to need roughly ten terabytes of space or 10,000 GB. If today 10 TB still seems like a lot, imagine 25 years ago. Enter Nalimov.

...


Contrary to previous versions of the bases which took on the names of their creators, de Man chose to call them Syzygy (pronounced Sih-Zih-Gee), which means a pair of connected or corresponding things. These new bases are not only best suited for multithread searches, but the full set also takes up a mere 150 GB. As such, they are now the de facto choice for use with an engine.

Sesse
Posts: 181
Joined: Mon Apr 30, 2018 9:51 pm
Contact:

Re: 7-men Syzygy attempt

Post by Sesse » Mon Aug 20, 2018 9:17 pm

Dann Corbit wrote:
Mon Aug 20, 2018 8:10 pm
Instead of 8 man files, I think the next logical project is the 7 man DTM with Ronald's new format.
Or 6-man DTM50, for that matter.

Actually, the next logical project is getting the probing code into Stockfish and Fathom.

Post Reply