Re: 7-men Syzygy attempt
Posted: Fri May 18, 2018 4:12 am
Lomonosov says draw.
Lomonosov says draw.
While stockfish evaluate it as +4. SF need that particular set for fortress detection.
We now know the 7-piece pawnless numbers:syzygy wrote: ↑Sat May 05, 2018 2:31 amThe pawn tables will probably take the ratio down.Sesse wrote:My estimation was just wild extrapolation. People talked about 10 TB for WDL, and for the current generated 7-man TBs, DTZ files are about 2.6x the size of WDL.
For 5-piece tables, the ratios DTZ/WDL are:For 6-piece tables:Code: Select all
WDL DTZ DTZ/WDL pawnless 68132288 287321600 4.217 pawnful 326710624 297447328 0.910 total 394842912 584768928 1.481
Going from 5 to 6, WDL size multiplied by 184.5. To keep 7-piece WDL below 10TB, the factor needs to be just below 151 (I am counting 1kb = 1024 ). But Lomonosov DTM size went up by a factor of about 150, I think, and WDL and DTZ should do better than that.Code: Select all
WDL DTZ DTZ/WDL pawnless 11896768096 30359010784 2.552 pawnful 60937299376 57032537776 0.936 total 72834067472 87391548560 1.200
Code: Select all
WDL DTZ DTZ/WDL
pawnless 1156756806976 2010529270464 1.738
Number of pawnless tables with 5, 6, 7 pieces: 60, 170, 396.
In any case, an impressive work!! Congratulations!!syzygy wrote: ↑Sun May 20, 2018 2:42 am http://kirill-kryukov.com/chess/nulp/results.html
The number of legal positions with 5, 6 and 7 pieces is 25912594054, 3787154440416 and 423836835667331.
Compression ratio for 5-piece WDL is 65.6276 positions per byte.
Compression ratio for 6-piece WDL is 51.9970 positions per byte.
For 7-piece WDL to remain under 8 TB, we need 48.1847 positions per byte.
In practice, building full 7-piece tables is going to take more than half a year on my two machines, with 8-piece at current speed it probably gonna take a century to build the full set. So I'd just wait for the hardware developments to reach that reasonable level first.syzygy wrote: ↑Mon May 14, 2018 9:29 pmIn theory it should work. In practice you'll need a very very big machine and a lot of patience.
It would probably be a good idea to rewrite the generator to use a more space-efficient indexing scheme during generation. Generation of the biggest table should then fit in about 41 TB (so let's say 48 TB) and most tables would need half of that or less. The current scheme would need 57.75 TB (so basically a 64 TB machine).
The bigger 8-piece tables will fill more than 2^32 64-byte blocks with compressed data, which means the format will need a relatively small extension to deal with them.
It could be interesting to generate a few well-chosen 8-piece tables to try to beat the current max DTZ record of 517 moves. That should be possible on one of your machines, but it would need a new generator. (And it is not clear that the max 8-piece DTZ exceeds the max 7-piece DTZ.)noobpwnftw wrote: ↑Mon May 21, 2018 4:09 pm In practice, building full 7-piece tables is going to take more than half a year on my two machines, with 8-piece at current speed it probably gonna take a century to build the full set. So I'd just wait for the hardware developments to reach that reasonable level first.
The DTM50 value of a position varies with the value of the 50-move counter, so that metric is rather complicated both for generation and for storage. At the moment I don't know if this is doable.While we wait, I'm getting extra storage ready for other interesting metrics(like DTM50) or variations(antichess, suiside) if there is interest, so after having full 7-piece DTZ50 on standard done, shall we try building stuff like those?
Thanks for all the information, I think you have just answered a lot of potential questions that may come forward .syzygy wrote: ↑Mon May 21, 2018 6:32 pmIt could be interesting to generate a few well-chosen 8-piece tables to try to beat the current max DTZ record of 517 moves. That should be possible on one of your machines, but it would need a new generator. (And it is not clear that the max 8-piece DTZ exceeds the max 7-piece DTZ.)noobpwnftw wrote: ↑Mon May 21, 2018 4:09 pm In practice, building full 7-piece tables is going to take more than half a year on my two machines, with 8-piece at current speed it probably gonna take a century to build the full set. So I'd just wait for the hardware developments to reach that reasonable level first.
The DTM50 value of a position varies with the value of the 50-move counter, so that metric is rather complicated both for generation and for storage. At the moment I don't know if this is doable.While we wait, I'm getting extra storage ready for other interesting metrics(like DTM50) or variations(antichess, suiside) if there is interest, so after having full 7-piece DTZ50 on standard done, shall we try building stuff like those?
Regular DTM tables would replicate the Lomonosov tables. I have a mostly finished DTM generator, but it needs too much memory now for pawns (it generates the whole table at once instead of by file, which was easier to implement since I compress these tables by rank). There may also be some performance problems that need to be addressed somehow. Total 7-piece DTM size would be of the order of the Lomonosov TBs (storing only the "smaller half" of them, like I do for DTZ tables, is not a real option since information for both sides is needed to handle promotions).
The problem of antichess/suicide chess is the sheer number of tables. There are 3570 pawnless 6-piece antichess tables and 4830 pawnful ones. I have not yet calculated the number of 7-piece antichess tables. The existing 6-piece tables at the lichess.ovh server have extra compression (essentially eliminating positions with a winning threat, i.e. a move that forces a losing capture by the other side), which would probably slow down generation of 7-piece tables too much. So those would need to be regenerated (approximately doubling their size, I think).
Atomic chess DTZ tables are probably the most similar to normal chess DTZ tables. The main complication is that kings can connect on the board in atomic chess (and also in antichess/suicide chess), which means that there are more than 462 king-king configurations. So the pawnless generator needs more memory.