How is work on 8-man tablebases progressing?

[Koistinen]

I think it would be nice to be able to compute 8-man tablebases on a normal computer with between 32 and 64 GB ram and perhaps 80 to 160 TB of hard disk in less than a year each. That way the work could be distributed among those interested.

5, 40, 185, 645, 1876, 4795, 11110, ...
/Urban

[phhnguyen]

I think it would be nice to be able to compute 8-man tablebases on a normal computer with between 32 and 64 GB ram and perhaps 80 to 160 TB of hard disk in less than a year each. That way the work could be distributed among those interested.

5, 40, 185, 645, 1876, 4795, 11110, ...
/Urban

Let me quote from CPW:

After the completion of the 7-man, many people started to be curious about the feasibility of building the 8-man. Ronald de Man estimated that the task requires computers with 64 TB RAM and 2000 TB hard disks[10] (cost about $640K and $40K respectively in 2020 [11]).

Your proposed hardware looks too weak!

Bojun Guo spent 5 months to generate 7-man Syzygy. I myself calculated that 8-man will be about 90-100 times larger than 7-man and I guess the creating time will close to that factor. Your estimation of "less than a year" seems to be too optimistic!

So far, Syzygy is the smallest one. If you mention other EGTBs/formats, x everything with a number, say, x10

[Koistinen]

Your proposed hardware looks too weak!

Bojun Guo spent 5 months to generate 7-man Syzygy. I myself calculated that 8-man will be about 90-100 times larger than 7-man and I guess the creating time will close to that factor. Your estimation of "less than a year" seems to be too optimistic!

So far, Syzygy is the smallest one. If you mention other EGTBs/formats, x everything with a number, say, x10

The hardware, storage and time is for each endgame class, just with 50 move rule and only white wins distance to resetting counter, not all in total.
Say for ♔♕♖♘-♚♛♝♞, you could use the result to look up if positions are won for white and how to win them but ♔♕♖♘-♚♛♝♝ would not be known until you did that computation, I.e. those would be two separate runs out of many to compute all 8-man.
(Storing it more compactly would remain a problem, compression as in Syzygy would be one way to solve it.)

Would it still be impressive as a proof of concept if someone were to manage that on, as you say, weak hardware?

[syzygy]

I think it would be nice to be able to compute 8-man tablebases on a normal computer with between 32 and 64 GB ram and perhaps 80 to 160 TB of hard disk in less than a year each. That way the work could be distributed among those interested.

5, 40, 185, 645, 1876, 4795, 11110, ...
/Urban

Let me quote from CPW:

After the completion of the 7-man, many people started to be curious about the feasibility of building the 8-man. Ronald de Man estimated that the task requires computers with 64 TB RAM and 2000 TB hard disks[10] (cost about $640K and $40K respectively in 2020 [11]).

Your proposed hardware looks too weak!

I was probably talking about my generator. With a different algorithm the tables can be generated with much less RAM at the cost of much longer generation times.

Many 8-piece tables have already been generated (but, I believe, not stored) by Yakov Konoval and Marc Bourzutschky. These are all pawnless tables because for pawnful tables you first need to generate, and store, the relevant pawnless tables. In principle generating pawnful tables requires less RAM. However, in practice they are much harder to write a generator for, and as mentioned you need to store and access all the tables which can be reached via promotions. And in a distributed 8-men generation project, you need to solve the problem of distributing all the data.

In the end, the storage requirements make such a project too impractical. Even if you generate all of them in a data center, what are you going to do with them? How many people have the full 7-men set? [I guess one could set up a server to allow people to access them, as has been done for the 7-piece tables. I don't know if anyone would be willing to host a server with such storage requirements, but I should not be too quick to dismiss this possibility.]

By the time that storage requirements for 8-piece tables are manageable, requiring 64TB of RAM will probably be acceptable. (And for most tables this number can be reduced by quite a bit by rewriting the generator without completely abandoning the approach of "in RAM generation". I wrote my generator just to do 6-piece tables on a decent PC, and for that task taking into account "like-piece symmetry" wasn't worth the trouble. For 7-piece and 8-piece this is different.)

Bojun Guo spent 5 months to generate 7-man Syzygy. I myself calculated that 8-man will be about 90-100 times larger than 7-man and I guess the creating time will close to that factor. Your estimation of "less than a year" seems to be too optimistic!

He meant 1 year per table. So that would be a couple of thousand years to generate all of them if you do it sequentially. Of course many tables can be generated in parallel if enough people are involved and you solve the problem of distributing the generated tables.

[hgm]

In practice one often does not need all Pawnless tables that could in theory result from a pawnful end-game. In particular the winning side usually cannot afford the weak side to promote if it wants to convert the win. And when one is doing DTZ, and thus don't care how many moves it exactly takes to reach checkmate after the first promotion, the only thing you need to know is whether the position after this promotion is won. And usually you would not need any further Pawn moves to force such a win, as the just obtained Queen is perfectly able to finish the job on her own (even if this might not be the fastest possible method).

There might be some positions for which this is not true, in particular those where both side have multiple Pawns one step away from promotion. But such positions are almost never reachable through a sensible series of moves, as they would involve pushing other Pawns to 7th rank (and waiting for the opponent to do so as well) instead of simply promoting the Pawn that is already there. So what would be the point of having such irrelevant positions in an EGT, if the fact that it requires orders of magnitude more effort to solve those?

It is a mistake to consider a pawnful end-game as a single table. One should consider each Pawn slice a table on its own, only solve the relevant Pawn slices, and ignore the much more difficult irrelevant ones.

[towforce]

The best incentive ever to solve chess: save the planet from having to produce millions of unnecessary chips and many gigawatts of electricity just to get to 8-man!!!

[syzygy]

In practice one often does not need all Pawnless tables that could in theory result from a pawnful end-game. In particular the winning side usually cannot afford the weak side to promote if it wants to convert the win.

I don't know how many people would be interested in spending enormous resources to calculate tables which are merely approximately correct.

It is a mistake to consider a pawnful end-game as a single table. One should consider each Pawn slice a table on its own, only solve the relevant Pawn slices, and ignore the much more difficult irrelevant ones.

Why not simply not calculate the 8-piece tables at all?

[hgm]

I don't know how many people would be interested in spending enormous resources to calculate tables which are merely approximately correct.

But they would not be 'approximately correct'. They would be merely incomplete. By handicapping the strong side with some restrictions, like not being allowed to give black the opportunity to move a promoted Pawn, only draws would be suspect. And many of the Pawn slices would have no draw positions at all (other than those where the draw is reached through immediate conversion).

Why not simply not calculate the 8-piece tables at all?

Silly question, which you could ask for anything that is discussed on this forum. Why not refrain from any programming activity in the area of Chess? Choose live!

But sad as it might be, people like their computers to play strong Chess, and they think that end-game tables can improve the quality of their analysis, or at least speed it up. So they would like to have tables, even if these are just DTZ or WDL, and that includes 8-man tables. And they won't get those by 'simply not calculating them at all'...

[Viz]

The main problem is that they imho have no purpose.
If TB7 is actually usable by chess engines TB8 definitely is not usable with current technology, we will need some breakthrough in ultra-fast storage.

[noobpwnftw]

If there is any genuine determination behind what people say, then there should already be an independently verified 7-piece TB result.

So far I have not seen it. Things are often easier said than done, at least Ronald, me and few others got something done.