Stockfish syzygy "bug"

[Jouni]

[d]8/8/4p3/2bP4/8/8/p7/k2KN3 w - - 0 1

Analysis by Stockfish default:

1.Kc1 Be3+ 2.Kc2 e5 3.d6 e4 4.d7 Bg5 5.Ng2 Bf6 6.Ne3 Be7 7.Nc4 Bb4 8.d8Q Ba3 9.Nxa3 e3 10.Qd1#
+- (#10) Depth: 31/32 00:00:02 3985kN

Analysis by Stockfish syzygy (5 piece):

1.Kc1 Be3+ 2.Kc2 exd5 3.Nd3 d4 4.Nc5 d3+ 5.Nxd3
+- (123.52) Depth: 39/30 00:00:33 30268kN, tb=1236242

Ronald, can this be fixed without slowing engine?

[Marek Soszynski]

Jouni, try the position with the latest Komodo too.

[ZirconiumX]

[d]8/8/4p3/2bP4/8/8/p7/k2KN3 w - - 0 1

Analysis by Stockfish default:

1.Kc1 Be3+ 2.Kc2 e5 3.d6 e4 4.d7 Bg5 5.Ng2 Bf6 6.Ne3 Be7 7.Nc4 Bb4 8.d8Q Ba3 9.Nxa3 e3 10.Qd1#
+- (#10) Depth: 31/32 00:00:02 3985kN

Analysis by Stockfish syzygy (5 piece):

1.Kc1 Be3+ 2.Kc2 exd5 3.Nd3 d4 4.Nc5 d3+ 5.Nxd3
+- (123.52) Depth: 39/30 00:00:33 30268kN, tb=1236242

Ronald, can this be fixed without slowing engine?

I think this is because Syzygy is DTZ rather than DTM.

Matthew: out

[syzygy]

It is not a bug. Please don't create unnecessary confusion.

Black can simply force a 5-piece position before white can mate.

[zullil]

It is not a bug. Please don't create unnecessary confusion.

Black can simply force a 5-piece position before white can mate.

[d]8/8/4p3/3P4/8/4b3/p1K5/k3N3 b - - 3 2

With 6-man syzygy tablebases, here Stockfish immediately avoids the capture exd5 (I assume because it yields a lost tablebase position) but does not arrive at the mate-in-8 score that the default Stockfish produces. Instead, the evaluation is:

Code: Select all

info depth 54 seldepth 21 score cp -12353 nodes 21635843094 nps 33291700 tbhits 129009729 time 649887 multipv 1 pv e6e5 d5d6 e5e4 d6d7 e3d2 d7d8q d2e1

Why does this occur? Just trying to understand these new tablebases and how Stockfish is using them.

[syzygy]

Why does this occur?

Black can simply force a 5-piece position before white can mate.

Same for a 6-piece position when using 6-piece TBs.

[ernest]

Analysis by Stockfish default:

1.Kc1 Be3+ 2.Kc2 e5 3.d6 e4 4.d7 Bg5 5.Ng2 Bf6 6.Ne3 Be7 7.Nc4 Bb4 8.d8Q Ba3 9.Nxa3 e3 10.Qd1#
+- (#10) Depth: 31/32 00:00:02 3985kN

Analysis by Stockfish syzygy (5 piece):

1.Kc1 Be3+ 2.Kc2 exd5 3.Nd3 d4 4.Nc5 d3+ 5.Nxd3
+- (123.52) Depth: 39/30 00:00:33 30268kN, tb=1236242

Ronald, can this be fixed without slowing engine?

I don't understand what you are complaining about...
In both cases, the move is Kc1 !

[mjlef]

[d]8/8/4p3/2bP4/8/8/p7/k2KN3 w - - 0 1

Analysis by Stockfish default:

1.Kc1 Be3+ 2.Kc2 e5 3.d6 e4 4.d7 Bg5 5.Ng2 Bf6 6.Ne3 Be7 7.Nc4 Bb4 8.d8Q Ba3 9.Nxa3 e3 10.Qd1#
+- (#10) Depth: 31/32 00:00:02 3985kN

Analysis by Stockfish syzygy (5 piece):

1.Kc1 Be3+ 2.Kc2 exd5 3.Nd3 d4 4.Nc5 d3+ 5.Nxd3
+- (123.52) Depth: 39/30 00:00:33 30268kN, tb=1236242

Ronald, can this be fixed without slowing engine?

I am needing to explain this a lot to Komodo customers too. THi is not a bug. It is how Syzygy tablebases work.

The Syzygy tablebases contain two sets of files. The DTZ (Distance to Zero) contain information on if a move is an immediate mate, draw or loss, and the number of half moves to conversion to another type of endgame (capture, promotion, pawn move). A second set has WLD (win loss draw) just tell you if the position is won, lost or is a draw. The WDL are normally used during the search. DTZ is used at the root normally. Neither knows the shortest path to mate in many positions. But DTZ at least lets you select a path towards the shorted conversion. DTZ will still preserve the win (or loss or draw), but it is not like older tablebases that tell you the number of moves to mate, loss or draw. To make the tablebase small and efficient, that information just is not in there.

The next effect is sometimes the program will make what look like strange moves, like capturing a piece instead of a shorter mate. But the score tells you the program sees a win, and it will find that win.

Some people do not install all of each set, and that can also look strange, and even cause the program to "lose" the win if whatever it converts to is not present on the computer. I suggest people have both full sets, to prevent these things from happening.

[petero2]

[d]8/8/4p3/2bP4/8/8/p7/k2KN3 w - - 0 1

Analysis by Stockfish default:

1.Kc1 Be3+ 2.Kc2 e5 3.d6 e4 4.d7 Bg5 5.Ng2 Bf6 6.Ne3 Be7 7.Nc4 Bb4 8.d8Q Ba3 9.Nxa3 e3 10.Qd1#
+- (#10) Depth: 31/32 00:00:02 3985kN

Analysis by Stockfish syzygy (5 piece):

1.Kc1 Be3+ 2.Kc2 exd5 3.Nd3 d4 4.Nc5 d3+ 5.Nxd3
+- (123.52) Depth: 39/30 00:00:33 30268kN, tb=1236242

Ronald, can this be fixed without slowing engine?

I am needing to explain this a lot to Komodo customers too. This is not a bug. It is how Syzygy tablebases work.

It is possible to keep searching after finding a forced TB win to try to find a faster mate though. The current development version of texel does that. When searching without TBs in the above position I get:

Code: Select all

info depth 22 score mate 10 time 2505 nodes 8306893 nps 3316124 pv d1c1 c5e3 c1c2 e6e5 d5d6 e5e4 d6d7 e3g5 e1g2 g5f6 g2e3 f6d8 e3f5 d8b6 d7d8q b6d8 f5d4 d8g5 d4b3

When using 6-men Syzygy TBs:

Code: Select all

info depth 22 score mate 10 time 1158 nodes 1367816 nps 1181188 tbhits 75896 pv d1c1 c5e3 c1c2 e6e5 d5d6 e5e4 d6d7 e3g5 e1g2 g5e7 g2f4 e7g5 d7d8q g5d8 f4e2 d8b6 e2c1 e4e3 c1b3

When using 6-men Syzygy TBs and 5-men Gaviota TBs:

Code: Select all

info depth 22 score mate 10 time 1184 nodes 1331638 nps 1124694 tbhits 72287 pv d1c1 c5e3 c1c2 e6e5 d5d6 e5e4 d6d7 e3g5 e1g2 g5e7 g2f4 e7g5 d7d8q g5d8 f4e2 d8b6 e2c1 e4e3 c1b3

The basic idea is that a tablebase probe can not just return an exact score, it can also return a lower or upper bound, just like a transposition table probe. A WDL/DTZ win/loss is then converted to a DTM bound using information about the longest possible win/loss in all dependent tablebase classes and information about the maximum number of remaining pawn moves.

This implementation adds a bit of complexity though, so whether it make sense or not depends on the goals of the engine developers. For stockfish one goal is small source code size and things that don't improve elo (such as analysis mode quality) is generally considered unimportant. Given those priorities the current stockfish/syzygy implementation makes perfect sense to me.

In texel I don't care much about code size and I do like to improve analysis mode quality when possible without affecting elo in a negative way, so I believe my implementation makes sense for texel.

The current development version of texel is available here: http://dl.dropboxusercontent.com/u/8968 ... el105a5.7z

[syzygy]

The basic idea is that a tablebase probe can not just return an exact score, it can also return a lower or upper bound, just like a transposition table probe.

Theoretically this is the "correct" approach. If a probe returns a "TB win", that is not an exact score, but a lower bound on the score.

But I would expect this to result in major search instability. Once beta exceeds the value of a "TB win", the TB probes become effectively useless. If a mate is found then everything is fine, but usually this won't be the case. (I have never actually tried this though, so maybe my concerns are unfounded.)

It is possible to address this instability by first limiting, at the root, beta to the "TB win" value. Once it is certain that the root position is at least a TB win, the search could be continued with (alpha, beta) = (TB win, INF), for example. If that fails low, a mate cannot be found yet and the next iteration should be started. But this would require too many changes to SF for what is intended to be an as simple as possible reference implementation.