Syzygy probing code: DTZ in some cursed endgames off by one?

[Michel]

All this craziness just to save 1 bit?

Probably you have measured the saved space and probably we are talking of some tens or hundred of MB..otherwise I really can't understand this design choice...

It not only to save one bit but also to improve compression. If you have fewer values then compression is more effective.

[syzygy]

Thanks. So it seems that in the end it is still a 2-ply search (which is needed anyway since only half the positions are stored) that saves the day... So maybe the clever reasoning that we lose only one ply is not necessary after all...

It would be nice is this were better documented. Perhaps I'll give it a shot...

The 1-ply search for a DTZ-optimal move turns into a 2-ply search if the DTZ table stores information for the side currently to move and remains a 1-ply search if the DTZ table stores information for the other side. A 2-ply search in the latter case (turning into a 3-ply search) will not give more precise information (well, unless it happens to bump into a mate or zeroing move, in the low-DTZ case).

Ply-accurate information could be obtained if the DTZ files stored tables with move-accurate information for both sides to move.

To come back to my previous comment, now let's consider the other case with white winning and DTZ storing information for black to move.

The DTZ table again encodes 0/1/2 as 0, 3/4 as 1, 5/6 as 2, etc.
So for black, real-dtz=3/4 corresponds to reported-dtz=3, real-dtz=5/6 to reported-dtz=5, real-dtz=7/8 to reported-dtz=7, etc.
For white, real-dtz=4/5 corresponds to reported-dtz=4, real-dtz=6/7 corresponds to reported-dtz=6, etc.

The critical situation is cnt50=93, real-dtz=7, reported-dtz=6. The question is if white can distinguish its best move leading to real-dtz=6 from a dangerous suboptimal move leading to real-dtz=7.

The move leading to real-dtz=6 will lead to a position with black to move with reported-dtz=5.

The move leading to real-dtz=7 will lead to a position black to move with reported-dtz=7.

So again all is fine.

[Michel]

Thanks again for the explanation. It is rather unexpected that with so much information loss (truncated dtz values for only half the positions) one can still reconstruct perfect play! Quite briljant!

[syzygy]

Thanks again for the explanation. It is rather unexpected that with so much information loss (truncated dtz values for only half the positions) one can still reconstruct perfect play! Quite briljant!

So a win found by the search from a non-TB position can always be converted.

Either real-dtz <= 99 on entering the DTZ table, in which case real-dtz + cnt50 <= 100 can be enforced by ensuring that reported-dtz + cnt50 <= 99 as moves are selected.

Or real-dtz = 100 on entering the DTZ table, in which case the DTZ table is ply-accurate and real-dtz == reported-dtz for all positions. So in that case the engine only needs to ensure that reported-dtz + cnt50 <= 100 as moves are selected.

Only if a DTZ table is entered in a cursed-win position (dtz > 100, cnt50 = 0) and the opponent after several moves "blunders" into a lost position with dtz < 100 but dtz + cnt50 == 100, is it possible that the engine fails to convert a winning position. That will be very rare, but such a game can of course be constructed.