How is work on 8-man tablebases progressing?

[syzygy]

I think I read somewhere that they are no longer accessible through the website, but with 140 TB of data I guess there is a chance that a part of all of it was lost.

It does look like they are still there:
https://tb7.chessok.com/probe/3/61

[Koistinen]

What do you expect to be improvements for 7-man?

Possible improvements I see are:

• Increased usability. Make it easy to implement use. This would help experimentation.
• More types, such as DTM, DTZ50, DTMZ50 available for all.
• Different file formats to support different usage goals, such as requiring less space.

[syzygy]

[*]More types, Zuch as DTM, DTZ50, DTMZ50 available for all.

DTZ50 is available if you mean distance-to-zeroing move while respecting the 50-move rule.

What is DTMZ50?
As I pointed out in another comment, DTM50 is not a practical metric (since it makes the value of a position dependent on the 50-move counter n a non-straigthforward way).

[syzygy]

I think I read somewhere that they are no longer accessible through the website, but with 140 TB of data I guess there is a chance that a part of all of it was lost.

It does look like they are still there:
https://tb7.chessok.com/probe/3/61

Aha, some years ago they were held hostage in a ransomware attack:
forum3/viewtopic.php?f=2&t=74046
So the hackers encrypted 140 TB of data? Duh.

It is not clear to me if they have been fully restored by now.

[Koistinen]

What is DTMZ50?
As I pointed out in another comment, DTM50 is not a practical metric (since it makes the value of a position dependent on the 50-move counter n a non-straigthforward way).

Yes, I meant DTM50, i.e. distance to mate for FIDE and chess.com chess.
It has different possibilities from the other metrics since there might be ways to compute it more quickly than 50x slower than DTZ50.
Likely can be stored in much less than 50x the space too. Anyway, it is cheaper than 8-man.

[syzygy]

What is DTMZ50?
As I pointed out in another comment, DTM50 is not a practical metric (since it makes the value of a position dependent on the 50-move counter n a non-straigthforward way).

Yes, I meant DTM50, i.e. distance to mate for FIDE and chess.com chess.
It has different possibilities from the other metrics since there might be ways to compute it more quickly than 50x slower than DTZ50.
Likely can be stored in much less than 50x the space too. Anyway, it is cheaper than 8-man.

True, 7-men DTM50 is cheaper than 8-men DTZ(50) and it should not be as bad a 50x 7-men DTZ50, but it is still quite a task. I agree it can be interesting to figure out (and implement) good ways to generate and compress them.

Btw, 100x DTZ50 might be the better comparison. There will be positions where having 1-ply more on the 50-move clock, say 3 ply left instead of 2 ply left, allows the winning side to capture a piece in 3 plies (leading to a quicker mate) instead of having to force the losing side to capture a piece on the previous ply (leading to a slower mate). Of course in practice the absolute value of the DTM50 value of the average board position will only jump up a few times as the 50-move counter goes from 0 (immediately after a capture or pawn move by the other side) to 99.

[Uri Blass]

What is DTMZ50?
As I pointed out in another comment, DTM50 is not a practical metric (since it makes the value of a position dependent on the 50-move counter n a non-straigthforward way).

Yes, I meant DTM50, i.e. distance to mate for FIDE and chess.com chess.
It has different possibilities from the other metrics since there might be ways to compute it more quickly than 50x slower than DTZ50.
Likely can be stored in much less than 50x the space too. Anyway, it is cheaper than 8-man.

True, 7-men DTM50 is cheaper than 8-men DTZ(50) and it should not be as bad a 50x 7-men DTZ50, but it is still quite a task. I agree it can be interesting to figure out (and implement) good ways to generate and compress them.

Btw, 100x DTZ50 might be the better comparison. There will be positions where having 1-ply more on the 50-move clock, say 3 ply left instead of 2 ply left, allows the winning side to capture a piece in 3 plies (leading to a quicker mate) instead of having to force the losing side to capture a piece on the previous ply (leading to a slower mate). Of course in practice the absolute value of the DTM50 value of the average board position will only jump up a few times as the 50-move counter goes from 0 (immediately after a capture or pawn move by the other side) to 99.

It may be interesting to have statistics about percentage of cases when having more than 100 plies on the 50 move clock allow faster mate and what is the biggest difference assuming we increase 100 plies to higher number of plies.

Can we get a correct distance to mate that include the 50 move rule at least for the 5 piece tablebases?


An easier question may be what is the shortest mate when you do not consider the 50 move rule that is practically a draw by the 50 move rule.
Do we have a mate in 51 that is practically a draw by the 50 move rule?

Edit:Maybe we can have a table of distance to mate assuming you have X plies on the 50 move clock for every X.
It can help analysis of games when the weaker side does not play perfectly and suddenly allow mate without a draw by the 50 move rule.

[Koistinen]

What is DTMZ50?
As I pointed out in another comment, DTM50 is not a practical metric (since it makes the value of a position dependent on the 50-move counter n a non-straigthforward way).

Yes, I meant DTM50, i.e. distance to mate for FIDE and chess.com chess.
It has different possibilities from the other metrics since there might be ways to compute it more quickly than 50x slower than DTZ50.
Likely can be stored in much less than 50x the space too. Anyway, it is cheaper than 8-man.

True, 7-men DTM50 is cheaper than 8-men DTZ(50) and it should not be as bad a 50x 7-men DTZ50, but it is still quite a task. I agree it can be interesting to figure out (and implement) good ways to generate and compress them.

Btw, 100x DTZ50 might be the better comparison. There will be positions where having 1-ply more on the 50-move clock, say 3 ply left instead of 2 ply left, allows the winning side to capture a piece in 3 plies (leading to a quicker mate) instead of having to force the losing side to capture a piece on the previous ply (leading to a slower mate). Of course in practice the absolute value of the DTM50 value of the average board position will only jump up a few times as the 50-move counter goes from 0 (immediately after a capture or pawn move by the other side) to 99.

Maybe 10x DTZ for computing it cause there would be the same number of retrograde steps but with 10 bits to load for the DTM part compared to 1 for DTZ. Then storage would naïvely be 10x100=1000 times larger, so that would be the most fun reducing.

[syzygy]

It may be interesting to have statistics about percentage of cases when having more than 100 plies on the 50 move clock allow faster mate and what is the biggest difference assuming we increase 100 plies to higher number of plies.

But the problem is that the 50-move rule may still play a role after the first capture or pawn move. In general you cannot replace the 50-move rule with a 60-move rule without basically recalculating all the tables.

Can we get a correct distance to mate that include the 50 move rule at least for the 5 piece tablebases?

This should certainly be feasible in terms of storage space.

And if one only cares about the DTM50 value of a position with 50-move counter set to 0 (so immediately after a capture from 6 into 5 pieces, or from 7 into 6), then such tables are easy to generate. The problem is that such tables would not allow you to find the full mate sequence.

Edit:Maybe we can have a table of distance to mate assuming you have X plies on the 50 move clock for every X.

If you restrict X to <= 100 plies, then this DTM50.

It can help analysis of games when the weaker side does not play perfectly and suddenly allow mate without a draw by the 50 move rule.

If the position is a cursed win, then just use DTZ(50+) and hope that the opponent makes a mistake. No real need to play for the shortest mate (whatever that means -- if you really just want the shortest mate hoping that the opponent will not manage to draw by the 50-move rule, then there is the conventional DTM metric which you can use).

Note that a cursed win can be a position which can be converted into another cursed win within the 50-move rule but then needs more than 50 moves for the next zeroing move. It would make sense to try to go for a "shortest second phase" (or third or fourth phase, depending on where the 50+ move sequence occurs), but DTM50 or a straightforward extension of DTM50 cannot do that.

[petero2]

An easier question may be what is the shortest mate when you do not consider the 50 move rule that is practically a draw by the 50 move rule.
Do we have a mate in 51 that is practically a draw by the 50 move rule?

I assume you are only interested in positions where the half-move clock (hmc) is 0, because otherwise there are lots of drawn positions with hmc=99 that would be mate in 2 without the 50-move rule.

I can only answer this question for 5-men positions, since I don't have access to 6-man DTM tables. The shortest loss for the side to move when ignoring the 50-move rule is "mated in 52 moves". An example is this position:
[d]8/8/8/8/8/1K5p/7N/1k2N3 b - - 0 1

The shortest win for the side to move when ignoring the 50-move rule is "mate in 53 moves". An example is this position:
[d]8/5N2/8/8/8/7p/1K5N/3k4 w - - 0 1