Syzygy implementations of top engines

[Nordlandia]

if DTM50 5-men can be shrinked to 1/2 or 1/4 of the uncompressed format. It will certainly be more attractive. I'm not saying it has to be done but it will surely be a nice format for mediocre hardware users. Not all testers consult 6-men for their tournament matches.

I let engines probe 5-men during games plus immediate 6-men adjudication. CuteChess 1.0.0 support 6-men adj and it works quite well.

Probing 5-men during search plus immediate 6-men adj is good alternative to probing 6-men during search.

[petero2]

I think that even if Texel's TB probing strategy is ideal, the result is not ideal because it also depends on how efficient the regular search is. I think that if texel's TB probing strategy was implemented in stockfish, it would likely get lower than 47.80 moves to mate on average for this test suite.

Ok, I thought that having 5-men DTZ50 and DTM can give perfect (shortest) play on 5-men at root wins. Well, if it depends on the search, then you are surely right, and I should have replaced "Ideal play" with "Texel probing play".

It certainly can happen, but I don't know if it actually did happen in your test. You could try letting texel search some of the hardest positions for significantly longer than 1 second to see if it finds shorter mates than it did after 1 second.

The problematic case happens when DTZ50 < 100ply but DTM > 100ply. Following DTZ50 blindly could lead to an unnecessary long path to mate, and following DTM blindly could lead to a draw by the 50-move rule. In these cases search is needed to find the best "mix" of the two strategies, and you could potentially need a very deep search to find the DTM50 optimal strategy.

I tested at 5s/move, but the average came a bit larger, 47.96 moves compared to 47.80 moves at 1s/move on these 225 positions.

Thanks, I have looked at the data more closely now. The following can be concluded:

In all positions where there is a difference between 1s and 5s the distance to mate is more than 50 moves. This is expected because when DTM<=50 gaviota probes immediately give the optimal solution without any need for search.

Texel's mate score announcements are correct (unless there is a zobrist hash collision that affects the root score) in the sense that if it claims "mate in X", the position is a forced mate in at most X moves. Therefore you can take the minimum announced mate distance for each position and conclude that ideal play for the whole set is no worse than 47.43 moves.

Texel's search is not very impressive for these kinds of positions since searching 5x longer did not cause any improvement.

[Evert]

The problematic case happens when DTZ50 < 100ply but DTM > 100ply. Following DTZ50 blindly could lead to an unnecessary long path to mate, and following DTM blindly could lead to a draw by the 50-move rule. In these cases search is needed to find the best "mix" of the two strategies, and you could potentially need a very deep search to find the DTM50 optimal strategy.

I think I'm missing something. Why can't you play the move with the shortest DTM that is a win according to DTZ50?

[hgm]

I guess there is no guarantee that a position with a small DTM isn't actually a position with a very large DTM when it has to work around the 50-move rule.

Cases where playing pure DTM would run into a 50-move draw should be pretty rare, though. It is a bit questionable whether always knowing the exect DTM, even if it is hundreds of moves away, is a sensible goal. In fact postponing the mate as long as you can is a losing strategy in the first place. DTZ50 play is optimal from a game-theoretical point of view. Most resistance to it comes from the cases where it forces a sacrificial conversion, e.g. in KQBNK to win via a lengthy KBNK. I guess there would be very few objections if an engine postponed the mate a few ply by going for a faster conversion that actually makes progress (in particular capturing or trading a defending piece).

So it would be much more useful to have, in addition to DTZ50, a 'DTP50', which would essentially be DTZ50 that would only use conversions that make progress. DTP50 play would look pretty natural, but it could miss some wins in cases where a sacrificial conversion must be forced to dodge the 50-move rule. (Which in particular would occur if the opponent blunders at high value of the ply counter.) You then use DTZ50 to guard against that, by playing the DTZ50 move if you are in a cursed DTP50 position.

[syzygy]

The problematic case happens when DTZ50 < 100ply but DTM > 100ply. Following DTZ50 blindly could lead to an unnecessary long path to mate, and following DTM blindly could lead to a draw by the 50-move rule. In these cases search is needed to find the best "mix" of the two strategies, and you could potentially need a very deep search to find the DTM50 optimal strategy.

I think I'm missing something. Why can't you play the move with the shortest DTM that is a win according to DTZ50?

You'll run into a repetition now and then. The DTM counter need not strictly decrease with your strategy, since the move by the winning side that lowers it might be a move that draws under the 50-move rule.

What you could do is follow the strategy you suggest except when there has been a repetition by either side since the last pawn move or capture. If there was a repetition, you just play DTZ50-optimal moves until you reach a pawn move or capture. This guarantees that the DTZ50 counter decreases on every half move starting from the repeated position until an irreversible pawn move or capture, which means that the once-repeated position will not be repeated another time.

[Ras]

You'll run into a repetition now and then.

Why not just evaluating a repetition with 0.00? The side that has advantage would automatically avoid it in this case.

And the other question, why a repetition at all? Obviously this requires a move sequence that at some point raises both DTM and DTZ, and why would the stronger side want this?

[syzygy]

You'll run into a repetition now and then.

Why not just evaluating a repetition with 0.00? The side that has advantage would automatically avoid it in this case.

And the other question, why a repetition at all?

When the repetition occurs it is too late to avoid the repetition. The error was made earlier.

Obviously this requires a move sequence that at some point raises both DTM and DTZ, and why would the stronger side want this?

There is not always a move that lowers both DTM and DTZ50.

[syzygy]

If you always pick the move with the shortest DTM from those moves that lower DTZ50, then there is no danger of running into a repetition. But that means you'll always move a pawn if there is a winning pawn move even if the pawn move does not contribute at all to lowering DTM.

I guess you could modify the strategy to simply pick lowest DTM if the lowest DTM needs fewer moves than are left on the 50-move counter.

[Ras]

There is not always a move that lowers both DTM and DTZ50.

I see, so when faced with that situation, the engine might be choosing an unlucky move, which later can lead to repetition.

Couldn't the engine choose by a priority list?

1) moves that lower both DTM and DTZ50.
2) if 1) is not available, but DTM is less than 50 (moves counter, I mean), choose DTM.
3) if neither 1) nor 2) are available, choose DTZ50.

[syzygy]

There is not always a move that lowers both DTM and DTZ50.

I see, so when faced with that situation, the engine might be choosing an unlucky move, which later can lead to repetition.

Couldn't the engine choose by a priority list?

1) moves that lower both DTM and DTZ50.
2) if 1) is not available, but DTM is less than 50, choose DTM.
3) if neither 1) nor 2) are available, choose DTZ50.

Yes, that seems to correspond to what I wrote a few seconds ago. (You'll have to modify 2) to look at the 50-move counter instead of assuming that 50 moves are left.)

Simply playing DTM-optimal moves (that stay within the 50-move rule) might give better-looking play though, except when the engine runs into a repetition.