Possible draw bug in Chessbase Houdini 4 GUI with Syzygy TB

[syzygy]

A GUI that is using DTM bases does not need to know which bases an engine is using.

If the GUI takes over from the engine, it does need to know that the engine does not use tablebases that know about the 50-move draw. Otherwise using DTM can turn a win into a 50-move draw, and it can turn a 50-move draw into a loss.

So if the engine uses Syzygybases, do not let the GUI play any moves. It won't go wrong often, but whenever it goes wrong a new "bug in Syzygybases" thread appears somewhere on the internet.

[syzygy]

I just thought it might help some folks to have this listed. (assuming I understand the above correctly, but again, anyone can correct me if wrong)

You understand it correctly, thanks for writing it down

[hgm]

A GUI that is using DTM bases does not need to know which bases an engine is using.

I think the discussion on Rybka forum shows this is wrong:

They showed a position that was a win according to DTM, and even was a win (taking the 50-move rule into account) when it was played out according to DTM by the GUI. But the problem was that DTM made the losing side defend stupidly. That side could have forced a 50-move draw, but because DTM did not know about it, it pushed a Pawn because it would delay the mate longer (but alas, without a stretch of more than 50 reversible moves before or after the Pawn push).

The engine would of course never have made that mistake, because it was playing by DTZ50. It could have avoided the problem in an earlier stage, where it could choose between a move that would even draw according to DTM, and one that was a 'phantom win' by DTM, but a 50-move draw in reality. The engine did not care which draw it would get: both were 100% certain draws, so it chose the latter. After which the GUI stepped in, turning the draw into a loss.

It is only the DTZ50 that tell you the true result with best play, so I think it would be logical to only accept that for GUI adjudication, or showing the best line. Of course with adjudication you would have the usual problem that you would not only often give non-tablebase users more than they deserve, but also DTM users. Because the latter could also blunder away drawn games, thinking they were lost, and playing to delay the mate, (which is unavoidable) rather than to delay the zeroing of the counter (which is also unavoidable, but is avoidable for the first 50 moves).

Of course you could adjudicate a theoretical DTZ50 draw only to engines that do not think the current position is a loss.

[syzygy]

Of course with adjudication you would have the usual problem that you would not only often give non-tablebase users more than they deserve, but also DTM users. Because the latter could also blunder away drawn games, thinking they were lost, and playing to delay the mate, (which is unavoidable) rather than to delay the zeroing of the counter (which is also unavoidable, but is avoidable for the first 50 moves).

As an example, here a game where black is playing for a 50-move draw using DTZ50 tables:
[pgn][FEN "8/8/8/6kp/2p5/2N2K2/8/3N4 w - - 0 1"]
1. Ne3 Kf6 2. Kf4 Ke6 3. Ng2 h4 4. Nxh4 Kd6 5. Ke3 Kc5 6. Kd2 Kb4 7. Kc2 Ka3
8. Ng2 Kb4 9. Ne3 Ka3 10. Ng4 Kb4 11. Nf6 Kc5 12. Nfd5 Kc6 13. Kd1 Kd6 14. Kd2
Ke6 15. Ke2 Kf5 16. Kf3 Ke5 17. Ke3 Kf5 18. Kd4 Ke6 19. Ke4 Kd6 20. Kf5 Kc6
21. Ke6 Kc5 22. Ke5 Kc6 23. Ne3 Kd7 24. Nf5 Kc6 25. Kd4 Kc7 26. Kc5 Kd7 27. Kd5
Kc7 28. Ke6 Kd8 29. Ng3 Kc7 30. Nge4 Kd8 31. Nf6 Kc8 32. Kd5 Kc7 33. Kc5 Kc8
34. Kc6 Kd8 35. Kd6 Kc8 36. Nfd5 Kb8 37. Nf4 Kb7 38. Ne6 Kb6 39. Kd5 Kb7
40. Kc5 Ka7 41. Kc6 Ka6 42. Nf4 Ka5 43. Nfd5 Ka6 44. Kd6 Ka7 45. Kc7 Ka6
46. Kc6 Ka5 47. Kc5 Ka6 48. Ne7 Kb7 49. Kd6 Kb8 50. Kd7 Ka8 51. Kc7 Ka7 52. Nf5
Ka6 53. Kc6 Ka5 54. Kc5 Ka6 55. Nd6 Ka5 56. Nb7 Ka6 57. Kc6 Ka7 58. Nc5 Kb8
59. Kd7 Ka8 60. Kc7 Ka7 61. Nb5 Ka8 62. Na4 c3 63. Nb6#[/pgn]
Black can claim a draw after 54.Kc5.
And now when black is playing using DTM (e.g. Nalimov) tables:
[pgn][FEN "8/8/8/6kp/2p5/2N2K2/8/3N4 w - - 0 1"]
1. Ne3 Kg6 2. Ng2 Kf5 3. Nh4 Ke5 4. Ne2 c3 5. Nxc3 Kd4 6. Nd1 Kd3 7. Ne3 Kd4
8. Kf4 Kd3 9. Nef5 Kc4 10. Ke5 Kc5 11. Nd6 Kc6 12. Ne4 Kd7 13. Kd5 Ke7 14. Ng5
Kd7 15. Ng6 h4 16. Nh3 Kc7 17. Ne5 Kb7 18. Kd6 Kb6 19. Nc6 Ka6 20. Nd4 Kb6
21. Kd5 Ka5 22. Kc5 Ka4 23. Kc4 Ka5 24. Nf3 Kb6 25. Ne5 Ka5 26. Nd7 Ka6 27. Kb4
Kb7 28. Kc5 Ka6 29. Nf8 Ka5 30. Ne6 Ka4 31. Nd4 Ka3 32. Kc4 Kb2 33. Kb4 Kc1
34. Kc3 Kd1 35. Kd3 Kc1 36. Ne6 Kb2 37. Nc5 Kc1 38. Na4 Kb1 39. Kd2 Ka2 40. Kc3
Kb1 41. Nc5 Kc1 42. Nb3 Kb1 43. Nd2 Ka2 44. Nc4 Kb1 45. Kd2 Ka2 46. Kc2 Ka1
47. Kb3 Kb1 48. Nb2 Kc1 49. Kc3 Kb1 50. Nd3 Ka2 51. Kb4 Kb1 52. Kb3 Ka1 53. Ne5
Kb1 54. Nf3 Kc1 55. Kc3 Kb1 56. Nf4 Ka1 57. Kb3 h3 58. Nh2 Kb1 59. Nd3 Ka1
60. Kc2 Ka2 61. Nb2 Ka1 62. Kc3 Ka2 63. Nc4 Kb1 64. Kd2 Ka1 65. Kc1 Ka2 66. Kc2
Ka1 67. Kb3 Kb1 68. Nd2 Kc1 69. Kc3 Kd1 70. Nb3 Ke1 71. Kd4 Ke2 72. Ke4 Ke1
73. Ke3 Kd1 74. Kd3 Ke1 75. Nd4 Kd1 76. Ne2 Ke1 77. Nc3 Kf2 78. Kd2 Kg2 79. Ke3
Kg3 80. Ne2 Kg2 81. Nd4 Kg3 82. Ndf3 Kg2 83. Nd2 Kg3 84. Ndf1 Kh4 85. Kf4 Kh5
86. Kf5 Kh6 87. Kf6 Kh5 88. Ne3 Kh6 89. Neg4 Kh7 90. Kf7 Kh8 91. Ne3 Kh7
92. Nf5 Kh8 93. Kg6 Kg8 94. Ng7 Kf8 95. Kf6 Kg8 96. Ne6 Kh7 97. Kg5 Kg8 98. Kg6
Kh8 99. Kf7 Kh7 100. Ng4 h2 101. Ng5 Kh8 102. Ne5 h1=Q 103. Ng6#[/pgn]
Black is lost after 1...Kg6.

[RJN]

A GUI should not use the de Man bases. They are "engine-only" bases.
A GUI should only give the TB path to engine.

This remark intrigues me. You know I am very skeptical about EGT use in GUIs in general, but would there be any reason why Syzygy EGTs are worse than others for use in a GUI? One legitimate use seems to speed up the '50-move dance' between two engines that both use EGTs, and would thus never be able to swindle each other. But I don't see off-hand why DTZ50 could not be used for that.

It seems that especially when you use DTM EGTs, you should never let the GUI play on behalve of the engine, when reaching the EGT stage. On Rybka forum there is an amusing thread where the GUI bungles a 50-move draw when it takes over from Houdini (who was using DTZ50 itself, and thus was counting on the draw). For this application DTZ50 seems the only acceptable solution in the GUI.

For sure Nalimov or orther DTM-tables should not be used by the GUI if the engine is using DTZ50 tables.

A GUI could use DTZ50 tables, but just playing straight DTZ50-optimal moves leads to rather ugly play. An engine can solve this by doing a search (using the tables to filter out bad moves). A GUI will not do that.

But however I look at it, I don't see the point in a GUI playing the moves instead of the engines. If you let the GUI take over, why not just adjudicate the game immediately. If both engines are using the same set of tables, that should not change the outcome. If one of the engines is not, it may change the outcome, but if the user decides that this is OK, why not.

I think it makes more sense for a GUI to only show TB information and not interfere with engine play. It would also be nice if GUIs offer a way to inspect the tables.

I think you did a fantastic job creating your Syzygy Bases, Ronald.

From what I see, most problems have been config issues, or misunderstanding on how your bases work. Here is a consolidation of "issues" I have seen on the message boards, but you are the final word, but anyone free to correct me. It took time to understand some of these things myself:

1) People using 6-piece set assume it also includes the 5-piece, and are not setting a path to include 5-piece. (probably the #1 misunderstanding)

2) People stating Syzygy is missing (for example) mate in 71, while other EGTB formats show a forced mate. However, they don't realize the other bases don't take the 50 move rule into account.

3) Enabling Syzygy bases for the GUI, and also in some cases not setting the path for the engine, and only the GUI. It seems that despite the ability to set a GUI path, this causes issues on most (all?) GUIs to date.

4) Misunderstanding the evaluation when a mate greater than 50 is reported (although this has been changed in later versions of SF)

5) Misunderstanding of the "swindle" behavior.

6) Path not being set correctly. A semi-colon (windows) or a period (Linux) needs to be used to separate paths. Also, I have seen posts where people have used spaces in the path before/after the semicolons.

7) Not using the DTZ files, then wondering why the position can't always be converted into a win/loss/draw.

8) In general, not understanding the concept of DTZ50.

9) The belief that a RAM disk will improve performance, and not understanding this does basically nothing, since the OS provides the caching. (this won't cause the Bases to operate incorrectly, but is unnecessary)

Of course, most of this is in your Overview link below, but many people don't seem to read it. I just thought it might help some folks to have this listed. (assuming I understand the above correctly, but again, anyone can correct me if wrong)

https://github.com/syzygy1/Stockfish

Forgot one, 32 bit systems can only use 5-piece, and not 6-piece.

BTW, I just read that on Rybka Forum, a reply by Ronald. I had heard that before, but don't use 32 bit myself. I don't claim to have discovered all of these items, just making a compilation.


Updated List, if anyone wants to C/P it anywhere: #4 also has added info

1) People using 6-piece set assume it also includes the 5-piece, and are not setting a path to include 5-piece. (probably the #1 misunderstanding)

2) People stating Syzygy is missing (for example) mate in 71, while other EGTB formats show a forced mate. However, they don't realize the other bases don't take the 50 move rule into account.

3) Enabling Syzygy bases for the GUI, and also in some cases not setting the path for the engine, and only the GUI. It seems that despite the ability to set a GUI path, this causes issues on most (all?) GUIs to date.

4) Misunderstanding the evaluation when a mate greater than 50 is reported (although this has been changed in later dev versions of SF. Houdini 4 will show +/-318.00 for win/loss, or 0.00 for draw)

5) Misunderstanding of the "swindle" behavior.

6) Path not being set correctly. A semi-colon (windows) or a period (Linux) needs to be used to separate paths. Also, I have seen posts where people have used spaces in the path before/after the semicolons.

7) Not using the DTZ files, then wondering why the position can't always be converted into a win/loss/draw.

8) In general, not understanding the concept of DTZ50.

9) The belief that a RAM disk will improve performance, and not understanding this does basically nothing, since the OS provides the caching. (this won't cause the Bases to operate incorrectly, but is unnecessary)

10) 32 bit systems can only use 5-piece. Memory constraints won't allow 6-piece.

[Matthias Gemuh]

A GUI that is using DTM bases does not need to know which bases an engine is using.

I think the discussion on Rybka forum shows this is wrong:

They showed a position that was a win according to DTM, and even was a win (taking the 50-move rule into account) when it was played out according to DTM by the GUI. But the problem was that DTM made the losing side defend stupidly. That side could have forced a 50-move draw, but because DTM did not know about it, it pushed a Pawn because it would delay the mate longer (but alas, without a stretch of more than 50 reversible moves before or after the Pawn push).

... .

Wow I overlooked that important detail.
Currently, ChessGUI does not adjudicate "mate in more than 50".
I shall look whether ChessGUI also avoids adjudicating "mate in less than 50" if the 50-move counter says 50-move-draw is theoretically possible.

[syzygy]

Of course with adjudication you would have the usual problem that you would not only often give non-tablebase users more than they deserve, but also DTM users. Because the latter could also blunder away drawn games, thinking they were lost, and playing to delay the mate, (which is unavoidable) rather than to delay the zeroing of the counter (which is also unavoidable, but is avoidable for the first 50 moves).

As an example, here a game where black is playing for a 50-move draw using DTZ50 tables:
[pgn][FEN "8/8/8/6kp/2p5/2N2K2/8/3N4 w - - 0 1"]
1. Ne3 Kf6 2. Kf4 Ke6 3. Ng2 h4 4. Nxh4 Kd6 5. Ke3 Kc5 6. Kd2 Kb4 7. Kc2 Ka3
8. Ng2 Kb4 9. Ne3 Ka3 10. Ng4 Kb4 11. Nf6 Kc5 12. Nfd5 Kc6 13. Kd1 Kd6 14. Kd2
Ke6 15. Ke2 Kf5 16. Kf3 Ke5 17. Ke3 Kf5 18. Kd4 Ke6 19. Ke4 Kd6 20. Kf5 Kc6
21. Ke6 Kc5 22. Ke5 Kc6 23. Ne3 Kd7 24. Nf5 Kc6 25. Kd4 Kc7 26. Kc5 Kd7 27. Kd5
Kc7 28. Ke6 Kd8 29. Ng3 Kc7 30. Nge4 Kd8 31. Nf6 Kc8 32. Kd5 Kc7 33. Kc5 Kc8
34. Kc6 Kd8 35. Kd6 Kc8 36. Nfd5 Kb8 37. Nf4 Kb7 38. Ne6 Kb6 39. Kd5 Kb7
40. Kc5 Ka7 41. Kc6 Ka6 42. Nf4 Ka5 43. Nfd5 Ka6 44. Kd6 Ka7 45. Kc7 Ka6
46. Kc6 Ka5 47. Kc5 Ka6 48. Ne7 Kb7 49. Kd6 Kb8 50. Kd7 Ka8 51. Kc7 Ka7 52. Nf5
Ka6 53. Kc6 Ka5 54. Kc5 Ka6 55. Nd6 Ka5 56. Nb7 Ka6 57. Kc6 Ka7 58. Nc5 Kb8
59. Kd7 Ka8 60. Kc7 Ka7 61. Nb5 Ka8 62. Na4 c3 63. Nb6#[/pgn]
Black can claim a draw after 54.Kc5.
And now when black is playing using DTM (e.g. Nalimov) tables:
[pgn][FEN "8/8/8/6kp/2p5/2N2K2/8/3N4 w - - 0 1"]
1. Ne3 Kg6 2. Ng2 Kf5 3. Nh4 Ke5 4. Ne2 c3 5. Nxc3 Kd4 6. Nd1 Kd3 7. Ne3 Kd4
8. Kf4 Kd3 9. Nef5 Kc4 10. Ke5 Kc5 11. Nd6 Kc6 12. Ne4 Kd7 13. Kd5 Ke7 14. Ng5
Kd7 15. Ng6 h4 16. Nh3 Kc7 17. Ne5 Kb7 18. Kd6 Kb6 19. Nc6 Ka6 20. Nd4 Kb6
21. Kd5 Ka5 22. Kc5 Ka4 23. Kc4 Ka5 24. Nf3 Kb6 25. Ne5 Ka5 26. Nd7 Ka6 27. Kb4
Kb7 28. Kc5 Ka6 29. Nf8 Ka5 30. Ne6 Ka4 31. Nd4 Ka3 32. Kc4 Kb2 33. Kb4 Kc1
34. Kc3 Kd1 35. Kd3 Kc1 36. Ne6 Kb2 37. Nc5 Kc1 38. Na4 Kb1 39. Kd2 Ka2 40. Kc3
Kb1 41. Nc5 Kc1 42. Nb3 Kb1 43. Nd2 Ka2 44. Nc4 Kb1 45. Kd2 Ka2 46. Kc2 Ka1
47. Kb3 Kb1 48. Nb2 Kc1 49. Kc3 Kb1 50. Nd3 Ka2 51. Kb4 Kb1 52. Kb3 Ka1 53. Ne5
Kb1 54. Nf3 Kc1 55. Kc3 Kb1 56. Nf4 Ka1 57. Kb3 h3 58. Nh2 Kb1 59. Nd3 Ka1
60. Kc2 Ka2 61. Nb2 Ka1 62. Kc3 Ka2 63. Nc4 Kb1 64. Kd2 Ka1 65. Kc1 Ka2 66. Kc2
Ka1 67. Kb3 Kb1 68. Nd2 Kc1 69. Kc3 Kd1 70. Nb3 Ke1 71. Kd4 Ke2 72. Ke4 Ke1
73. Ke3 Kd1 74. Kd3 Ke1 75. Nd4 Kd1 76. Ne2 Ke1 77. Nc3 Kf2 78. Kd2 Kg2 79. Ke3
Kg3 80. Ne2 Kg2 81. Nd4 Kg3 82. Ndf3 Kg2 83. Nd2 Kg3 84. Ndf1 Kh4 85. Kf4 Kh5
86. Kf5 Kh6 87. Kf6 Kh5 88. Ne3 Kh6 89. Neg4 Kh7 90. Kf7 Kh8 91. Ne3 Kh7
92. Nf5 Kh8 93. Kg6 Kg8 94. Ng7 Kf8 95. Kf6 Kg8 96. Ne6 Kh7 97. Kg5 Kg8 98. Kg6
Kh8 99. Kf7 Kh7 100. Ng4 h2 101. Ng5 Kh8 102. Ne5 h1=Q 103. Ng6#[/pgn]
Black is lost after 1...Kg6.

Thanks to user galen at Kirill's Endgame Tablebases discussion board, here is a DTM50- optimal continuation after 5...Nxc3:
[pgn][FEN "8/8/8/4k2p/7N/2N2K2/8/8 b - - 0 5"]
5. ... Kd4 6. Nd1 Kd3 7. Ne3 Kd4 8. Kf4 Kd3 9. Nef5 Kc4 10. Ke5 Kc5 11. Ng7 Kc4 12. Ne6 Kd3 13. Kf4 Kc3 14. Ke3 Kc4 15. Ke4 Kc3 16. Kd5 Kb4 17. Kd4 Kb5 18. Nd8 Kb4 19. Nc6+ Kb5 20. Kd5 Ka6 21. Nd4 Kb6 22. Kd6 Ka5 23. Kc5 Ka4 24. Nc6 Kb3 25. Kd4 Kc2 26. Na5 Kd2 27. Nb3+ Ke2 28. Ke4 Kf2 29. Nd4 Kg3 30. Ndf5+ Kg4 31. Nf3 h4 32. Ne3+ Kg3 33. Nf1+ Kf2 34. N3h2 Ke2 35. Kd4 Kd1 36. Kc3 Ke2 37. Kc2 Ke1 38. Kd3 Kd1 39. Ne3+ Kc1 40. Kc3 Kb1 41. Nf3 Kc1 42. Nd5 Kd1 43. Nf4 h3 44. Nh2 Kc1 45. Nd5 Kb1 46. Ne3 Kc1 47. Kc4 Kb2 48. Kb4 Ka1 49. Nc4 Ka2 50. Kc3
Kb1 51. Kd2 Ka1 52. Kc1 Ka2 53. Kc2 Ka1 54. Kb3 Kb1 55. Nd2+ Kc1 56. Kc3 Kd1
57. Nb3 Ke1 58. Kd4 Ke2 59. Ke4 Ke1 60. Ke3 Kd1 61. Kd3 Ke1 62. Nd4 Kd1 63.
Ne2 Ke1 64. Nc3 Kf2 65. Kd2 Kg2 66. Ke2 Kg3 67. Ke3 Kh4 68. Kf4 Kh5 69. Kf5
Kh6 70. Kf6 Kh5 71. Ne4 Kh4 72. Kf5 Kh5 73. Nd2 Kh4 74. Ndf1 Kh5 75. Ne3
Kh6 76. Kf6 Kh7 77. Nf5 Kg8 78. Ke7 Kh7 79. Kf7 Kh8 80. Kg6 Kg8 81. Ng7 Kf8
82. Kf6 Kg8 83. Ne6 Kh7 84. Kg5 Kg8 85. Kg6 Kh8 86. Kf7 Kh7 87. Ng4 h2 88.
Nf8+ Kh8 89. Ne5 h1=Q 90. Neg6#[/pgn]