My troubles with MultiPV and Syzygy in Stockfish 7

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by Laskos »

petero2 wrote:
hgm wrote:Indeed, a good DTZ format should really contain these numbers (the maximum DTM that DTZ=0 corresponds to) as part of the data they hold.

Of course if you don't want silly DTZ behavior, like sacrifycing all the excess material and then slowly but surely win the most difficult end-game that you can still win, no attempt to hide it can be entirely satisfactory, and it would be better to just not use DTZ. In a certain sense this is a self-inflicted problem. It is not obvious that trading extra disk space for the need to make extra searches eventually leads to more efficient use of resources.
In texel I continue searching after finding a forced TB win, which cures many of the "silly DTZ behavior" problems, but not all of them.

Not using DTZ is currently not a good solution I think. Using pure DTM is inferior in terms of how much it would score in real games, because it ignores the 50-move draw rule. Using DTM50 would be best, except for the fact that there are no publicly available 6-men DTM50 tables, and even if there were, they would probably be far too big to fit in a typical SSD drive.

This page contains lots of interesting information about DTM50 tablebases.
These aesthetic considerations shouldn't affect the actual game-play. It seems both the new Texel and Texel 1.05 miss Syzygy 5-men Wins at root against SF 7 Syzygy in about 1% of cases. Also they can lose Draws in 3-4% of cases. The games are not deterministic, so it's hard to replicate the behavior. Just examples of games at 10''+0.1'' where Texel misses the theoretical outcome:

Win at root (mate in 23):
[pgn][White "Texel 1.06 Syzygy"]
[Black "SF7 Syzygy"]
[Result "1/2-1/2"]
[SetUp "1"]
[FEN "8/8/3K2k1/8/8/2n5/8/6BB w - - 0 0"]

1. Bf2 Kf5 2. Be1 Ne2 3. Kd5 Nf4+ 4. Kd4 Ne6+ 5. Ke3 Ng7 6. Bg2 Ne8 7. Bc3 Ke6 8. Kd4 Ng7 9. Bf3 Ne8 10. Bg4+ Ke7 11. Ke5 Ng7 12. Bd1 Kf7 13. Bb3+ Kg6 14. Kd6 Nf5+ 15. Kd7 Ng7 16. Bc2+ Kf7 17. Bd4 Nh5 18. Bb3+ Kg6 19. Ke7 Kf5 20. Bc2+ Kf4 21. Bb6 Ng3 22. Bc7+ Kf3 23. Bd1+ Kf2 24. Ke6 Ne2 25. Bb6+ Ke1 26. Bc2 Kd2 27. Bf5 Kc3 28. Kd5 Kd2 29. Bd8 Ng3 30. Bg6 Ke3 31. Ke5 Nf1 32. Be8 Nd2 33. Bb6+ Kd3 34. Bb5+ Kc3 35. Ba5+ Kc2 36. Kd5 Nf3 37. Ke4 Nd2+ 38. Ke3 Nb3 39. Ba4 Kb2 40. Bb4 Nc1 41. Kd2 Na2 42. Be7 Nc3 43. Bf6 Ka3 44. Bc6 Na4 45. Be7+ Kb3 46. Bd5+ Kb2 47. Be6 Nb6 48. Ke3 Kc2 49. Ba2 Na4 50. Bc4 Nb2 51. Bf1 1/2-1/2[/pgn]

Draw at root:
[pgn][White "SF7 Syzygy"]
[Black "Texel 1.06 Syzygy"]
[Result "1-0"]
[SetUp "1"]
[FEN "1R6/8/4k1r1/8/8/8/5K2/6B1 w - - 0 0"]

1. Bh2 Rg4 2. Kf3 Rd4 3. Bf4 Rc4 4. Ra8 Rc2 5. Ra5 Rc6 6. Be3 Rc3 7. Ke4 Ke7 8. Bd4 Rc2 9. Ra7+ Kd6 10. Be5+ Ke6 11. Ra6+ Kd7 12. Kd5 Re2 13. Ra7+ Kd8 14. Ke6 Rc2 15. Ra8+ Rc8 16. Ra4 Rc6+ 17. Kd5 Rc2 18. Bd6 Rg2 19. Kc6 Ke8 20. Rf4 Rg8 21. Rf3 Kd8 22. Rf2 Ke8 23. Rf1 Rh8 24. Rf3 Rg8 25. Rf6 Rh8 26. Rf1 Rg8 27. Rf2 Kd8 28. Rf7 Ke8 29. Re7+ Kf8 30. Rb7+ Ke8 31. Be5 Rg6+ 32. Kd5 Rh6 33. Rg7 Kd8 34. Bf4 Ra6 35. Bd6 Kc8 36. Rc7+ Kb8 37. Rh7+ Kc8 38. Rg7 Ra5+ 39. Ke6 Rh5 40. Bf4 Rc5 41. Kd6 Rc2 42. Be3 Ra2 43. Kd5 Rc2 44. Bc5 Kd8 45. Bd6 Re2 46. Bf4 Re1 47. Be5 Rd1+ 48. Ke6 Kc8 49. Rc7+ Kd8 50. Rc2 Ra1 51. Bxa1 Ke8 52. Rc8+ 1-0[/pgn]The second game shows a Rook sack at move 50.
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by Guenther »

Laskos wrote:These aesthetic considerations shouldn't affect the actual game-play. It seems both the new Texel and Texel 1.05 miss Syzygy 5-men Wins at root against SF 7 Syzygy in about 1% of cases. Also they can lose Draws in 3-4% of cases. The games are not deterministic, so it's hard to replicate the behavior. Just examples of games at 10''+0.1'' where Texel misses the theoretical outcome:

Draw at root:
[pgn][White "SF7 Syzygy"]
[Black "Texel 1.06 Syzygy"]
[Result "1-0"]
[SetUp "1"]
[FEN "1R6/8/4k1r1/8/8/8/5K2/6B1 w - - 0 0"]

1. Bh2 Rg4 2. Kf3 Rd4 3. Bf4 Rc4 4. Ra8 Rc2 5. Ra5 Rc6 6. Be3 Rc3 7. Ke4 Ke7 8. Bd4 Rc2 9. Ra7+ Kd6 10. Be5+ Ke6 11. Ra6+ Kd7 12. Kd5 Re2 13. Ra7+ Kd8 14. Ke6 Rc2 15. Ra8+ Rc8 16. Ra4 Rc6+ 17. Kd5 Rc2 18. Bd6 Rg2 19. Kc6 Ke8 20. Rf4 Rg8 21. Rf3 Kd8 22. Rf2 Ke8 23. Rf1 Rh8 24. Rf3 Rg8 25. Rf6 Rh8 26. Rf1 Rg8 27. Rf2 Kd8 28. Rf7 Ke8 29. Re7+ Kf8 30. Rb7+ Ke8 31. Be5 Rg6+ 32. Kd5 Rh6 33. Rg7 Kd8 34. Bf4 Ra6 35. Bd6 Kc8 36. Rc7+ Kb8 37. Rh7+ Kc8 38. Rg7 Ra5+ 39. Ke6 Rh5 40. Bf4 Rc5 41. Kd6 Rc2 42. Be3 Ra2 43. Kd5 Rc2 44. Bc5 Kd8 45. Bd6 Re2 46. Bf4 Re1 47. Be5 Rd1+ 48. Ke6 Kc8 49. Rc7+ Kd8 50. Rc2 Ra1 51. Bxa1 Ke8 52. Rc8+ 1-0[/pgn]The second game shows a Rook sack at move 50.
I don't understand the second game. Black's move Ra1 was ply 100 of
the draw counter, which means it could have declared a draw with that move?
When you started the game from FEN/EPD what draw move counter was set?
May be it was simply ignored or not set at all by the involved software.

Guenther
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by syzygy »

jdart wrote:I think possibly you could get n lines in analysis if you allowed all moves to be searched but always ordered moves so that the win/draw preserving ones are first.
Yes, that is possible, but it would require more invasive changes to SF's root search. And in the normal case it is a bit silly to search moves that you already know are not going to be played (but this is not critical).

I guess root moves could be assigned a two-component score: the TB result and the search result. Sorting would first look at the TB component, then at the search component. Only the TB result would be shown to the user, unless the search result is a mate score.

Maybe it would not needlessly complicate the code. I might have a look.
User avatar
hgm
Posts: 28395
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by hgm »

But this is of course what naturally happens if the evaluation adds a big constant for EGT wins...
petero2
Posts: 729
Joined: Mon Apr 19, 2010 7:07 pm
Location: Sweden
Full name: Peter Osterlund

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by petero2 »

Laskos wrote:
petero2 wrote:
hgm wrote:Indeed, a good DTZ format should really contain these numbers (the maximum DTM that DTZ=0 corresponds to) as part of the data they hold.

Of course if you don't want silly DTZ behavior, like sacrifycing all the excess material and then slowly but surely win the most difficult end-game that you can still win, no attempt to hide it can be entirely satisfactory, and it would be better to just not use DTZ. In a certain sense this is a self-inflicted problem. It is not obvious that trading extra disk space for the need to make extra searches eventually leads to more efficient use of resources.
In texel I continue searching after finding a forced TB win, which cures many of the "silly DTZ behavior" problems, but not all of them.

Not using DTZ is currently not a good solution I think. Using pure DTM is inferior in terms of how much it would score in real games, because it ignores the 50-move draw rule. Using DTM50 would be best, except for the fact that there are no publicly available 6-men DTM50 tables, and even if there were, they would probably be far too big to fit in a typical SSD drive.

This page contains lots of interesting information about DTM50 tablebases.
These aesthetic considerations shouldn't affect the actual game-play. It seems both the new Texel and Texel 1.05 miss Syzygy 5-men Wins at root against SF 7 Syzygy in about 1% of cases. Also they can lose Draws in 3-4% of cases. The games are not deterministic, so it's hard to replicate the behavior. Just examples of games at 10''+0.1'' where Texel misses the theoretical outcome:

Win at root (mate in 23):
[pgn][White "Texel 1.06 Syzygy"]
[Black "SF7 Syzygy"]
[Result "1/2-1/2"]
[SetUp "1"]
[FEN "8/8/3K2k1/8/8/2n5/8/6BB w - - 0 0"]

1. Bf2 Kf5 2. Be1 Ne2 3. Kd5 Nf4+ 4. Kd4 Ne6+ 5. Ke3 Ng7 6. Bg2 Ne8 7. Bc3 Ke6 8. Kd4 Ng7 9. Bf3 Ne8 10. Bg4+ Ke7 11. Ke5 Ng7 12. Bd1 Kf7 13. Bb3+ Kg6 14. Kd6 Nf5+ 15. Kd7 Ng7 16. Bc2+ Kf7 17. Bd4 Nh5 18. Bb3+ Kg6 19. Ke7 Kf5 20. Bc2+ Kf4 21. Bb6 Ng3 22. Bc7+ Kf3 23. Bd1+ Kf2 24. Ke6 Ne2 25. Bb6+ Ke1 26. Bc2 Kd2 27. Bf5 Kc3 28. Kd5 Kd2 29. Bd8 Ng3 30. Bg6 Ke3 31. Ke5 Nf1 32. Be8 Nd2 33. Bb6+ Kd3 34. Bb5+ Kc3 35. Ba5+ Kc2 36. Kd5 Nf3 37. Ke4 Nd2+ 38. Ke3 Nb3 39. Ba4 Kb2 40. Bb4 Nc1 41. Kd2 Na2 42. Be7 Nc3 43. Bf6 Ka3 44. Bc6 Na4 45. Be7+ Kb3 46. Bd5+ Kb2 47. Be6 Nb6 48. Ke3 Kc2 49. Ba2 Na4 50. Bc4 Nb2 51. Bf1 1/2-1/2[/pgn]

Draw at root:
[pgn][White "SF7 Syzygy"]
[Black "Texel 1.06 Syzygy"]
[Result "1-0"]
[SetUp "1"]
[FEN "1R6/8/4k1r1/8/8/8/5K2/6B1 w - - 0 0"]

1. Bh2 Rg4 2. Kf3 Rd4 3. Bf4 Rc4 4. Ra8 Rc2 5. Ra5 Rc6 6. Be3 Rc3 7. Ke4 Ke7 8. Bd4 Rc2 9. Ra7+ Kd6 10. Be5+ Ke6 11. Ra6+ Kd7 12. Kd5 Re2 13. Ra7+ Kd8 14. Ke6 Rc2 15. Ra8+ Rc8 16. Ra4 Rc6+ 17. Kd5 Rc2 18. Bd6 Rg2 19. Kc6 Ke8 20. Rf4 Rg8 21. Rf3 Kd8 22. Rf2 Ke8 23. Rf1 Rh8 24. Rf3 Rg8 25. Rf6 Rh8 26. Rf1 Rg8 27. Rf2 Kd8 28. Rf7 Ke8 29. Re7+ Kf8 30. Rb7+ Ke8 31. Be5 Rg6+ 32. Kd5 Rh6 33. Rg7 Kd8 34. Bf4 Ra6 35. Bd6 Kc8 36. Rc7+ Kb8 37. Rh7+ Kc8 38. Rg7 Ra5+ 39. Ke6 Rh5 40. Bf4 Rc5 41. Kd6 Rc2 42. Be3 Ra2 43. Kd5 Rc2 44. Bc5 Kd8 45. Bd6 Re2 46. Bf4 Re1 47. Be5 Rd1+ 48. Ke6 Kc8 49. Rc7+ Kd8 50. Rc2 Ra1 51. Bxa1 Ke8 52. Rc8+ 1-0[/pgn]The second game shows a Rook sack at move 50.
Thanks for testing. However, for the second game, it should have been a draw after move 50, but since UCI engines can not claim a draw, they rely on the GUI to claim the draw for them. For some reason the GUI did not do that in this case. Maybe setting the full move counter to 0 confuses the GUI.

I can not reproduce the problem in the first game. Do you have a log file? Already the first move Bf2 gives up the win. On my computer texel finds the best move after 2ms at depth 2 after 112 nodes and 12 TB hits.

Code: Select all

uci
id name Texel 1.06a43 64-bit
id author Peter Osterlund
option name UCI_EngineAbout type string default Texel 1.06a43 64-bit by Peter Osterlund, see http://web.comhem.se/petero2home/javachess/index.html#texel
option name Hash type spin default 16 min 1 max 524288
option name OwnBook type check default false
option name BookFile type string default 
option name Ponder type check default true
option name UCI_AnalyseMode type check default false
option name UCI_Opponent type string default 
option name Strength type spin default 1000 min 0 max 1000
option name Threads type spin default 1 min 1 max 64
option name MultiPV type spin default 1 min 1 max 256
option name UseNullMove type check default true
option name GaviotaTbPath type string default 
option name GaviotaTbCache type spin default 1 min 1 max 2047
option name SyzygyPath type string default 
option name MinProbeDepth type spin default 1 min 0 max 100
option name Clear Hash type button
option name BufferTime type spin default 1000 min 1 max 10000
uciok
setoption name SyzygyPath value /home/petero/chess/rtb/wdl:/home/petero/chess/rtb/dtz
info string Found 145 syzygy tablebases
position fen 8/8/3K2k1/8/8/2n5/8/6BB w - - 0 1
go wtime 10000 winc 100 btime 10000 binc 100
info depth 1
info depth 1 score cp 99 time 1 nodes 3 nps 3000 pv g1f2
info depth 2
info depth 2 score cp 35 upperbound time 2 nodes 100 nps 50000 tbhits 1 pv g1f2
info depth 2 score cp 35 time 2 nodes 101 nps 50500 tbhits 1 pv g1f2
info depth 2 score mate 31 lowerbound time 2 nodes 112 nps 56000 tbhits 12 pv d6e5
info depth 2 score cp 96 time 2 nodes 129 nps 64500 tbhits 13 pv d6e5 g6g5
info depth 2 score mate 45 lowerbound time 2 nodes 131 nps 65500 tbhits 15 pv d6e6
info depth 2 score cp 96 time 2 nodes 135 nps 67500 tbhits 16 pv d6e5 g6g5
info depth 3
info depth 3 score mate 31 lowerbound time 3 nodes 143 nps 47666 tbhits 24 pv d6e5 g6g5
info depth 3 score mate 31 time 3 nodes 228 nps 76000 tbhits 39 pv d6e5 g6f7
info depth 4
info depth 4 score mate 31 time 3 nodes 290 nps 96666 tbhits 47 pv d6e5 g6f7
info depth 5
info depth 5 score mate 31 time 5 nodes 531 nps 106200 tbhits 104 pv d6e5 g6f7 g1c5 f7g6
info depth 6
info depth 6 score mate 31 time 7 nodes 789 nps 112714 tbhits 167 pv d6e5 g6f7 g1c5 f7g6
info depth 7
info depth 7 score mate 31 time 12 nodes 1701 nps 141750 tbhits 330 pv d6e5 g6f7 g1c5 f7g6
info depth 8
info depth 8 score mate 31 time 18 nodes 3224 nps 179111 tbhits 570 pv d6e5 g6f7 g1c5 f7g6
info depth 9
info depth 9 score mate 31 time 39 nodes 8948 nps 229435 tbhits 1537 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7
info depth 10
info depth 10 score mate 31 time 95 nodes 24267 nps 255442 tbhits 4287 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4
info depth 11
info depth 11 score mate 31 time 152 nodes 38719 nps 254730 tbhits 7012 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4
info depth 12
info depth 12 score mate 31 time 352 nodes 98216 nps 279022 tbhits 17328 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4 e5d6 a7b5 d6c5 b5a7
info depth 13
info nodes 153863 nps 282317 tbhits 27760 time 545
bestmove d6e5 ponder g6f7
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by syzygy »

hgm wrote:But this is of course what naturally happens if the evaluation adds a big constant for EGT wins...
The evaluation?
So you still don't know what this is about.

Look, if you want to build your own marvelous system, that is fine with me. Go ahead and build something that actually works. But your imaginary solutions have little to do with what this thread is about.
User avatar
hgm
Posts: 28395
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by hgm »

syzygy wrote:The evaluation?
So you still don't know what this is about.
What a strange conclusion. You don't seem to have understood much of what I wrote...

That you can very precisely explain why it works so poorly doesn't alter the fact that it works poorly.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by Laskos »

petero2 wrote:
Laskos wrote:
petero2 wrote:
hgm wrote:Indeed, a good DTZ format should really contain these numbers (the maximum DTM that DTZ=0 corresponds to) as part of the data they hold.

Of course if you don't want silly DTZ behavior, like sacrifycing all the excess material and then slowly but surely win the most difficult end-game that you can still win, no attempt to hide it can be entirely satisfactory, and it would be better to just not use DTZ. In a certain sense this is a self-inflicted problem. It is not obvious that trading extra disk space for the need to make extra searches eventually leads to more efficient use of resources.
In texel I continue searching after finding a forced TB win, which cures many of the "silly DTZ behavior" problems, but not all of them.

Not using DTZ is currently not a good solution I think. Using pure DTM is inferior in terms of how much it would score in real games, because it ignores the 50-move draw rule. Using DTM50 would be best, except for the fact that there are no publicly available 6-men DTM50 tables, and even if there were, they would probably be far too big to fit in a typical SSD drive.

This page contains lots of interesting information about DTM50 tablebases.
These aesthetic considerations shouldn't affect the actual game-play. It seems both the new Texel and Texel 1.05 miss Syzygy 5-men Wins at root against SF 7 Syzygy in about 1% of cases. Also they can lose Draws in 3-4% of cases. The games are not deterministic, so it's hard to replicate the behavior. Just examples of games at 10''+0.1'' where Texel misses the theoretical outcome:

Win at root (mate in 23):
[pgn][White "Texel 1.06 Syzygy"]
[Black "SF7 Syzygy"]
[Result "1/2-1/2"]
[SetUp "1"]
[FEN "8/8/3K2k1/8/8/2n5/8/6BB w - - 0 0"]

1. Bf2 Kf5 2. Be1 Ne2 3. Kd5 Nf4+ 4. Kd4 Ne6+ 5. Ke3 Ng7 6. Bg2 Ne8 7. Bc3 Ke6 8. Kd4 Ng7 9. Bf3 Ne8 10. Bg4+ Ke7 11. Ke5 Ng7 12. Bd1 Kf7 13. Bb3+ Kg6 14. Kd6 Nf5+ 15. Kd7 Ng7 16. Bc2+ Kf7 17. Bd4 Nh5 18. Bb3+ Kg6 19. Ke7 Kf5 20. Bc2+ Kf4 21. Bb6 Ng3 22. Bc7+ Kf3 23. Bd1+ Kf2 24. Ke6 Ne2 25. Bb6+ Ke1 26. Bc2 Kd2 27. Bf5 Kc3 28. Kd5 Kd2 29. Bd8 Ng3 30. Bg6 Ke3 31. Ke5 Nf1 32. Be8 Nd2 33. Bb6+ Kd3 34. Bb5+ Kc3 35. Ba5+ Kc2 36. Kd5 Nf3 37. Ke4 Nd2+ 38. Ke3 Nb3 39. Ba4 Kb2 40. Bb4 Nc1 41. Kd2 Na2 42. Be7 Nc3 43. Bf6 Ka3 44. Bc6 Na4 45. Be7+ Kb3 46. Bd5+ Kb2 47. Be6 Nb6 48. Ke3 Kc2 49. Ba2 Na4 50. Bc4 Nb2 51. Bf1 1/2-1/2[/pgn]

Draw at root:
[pgn][White "SF7 Syzygy"]
[Black "Texel 1.06 Syzygy"]
[Result "1-0"]
[SetUp "1"]
[FEN "1R6/8/4k1r1/8/8/8/5K2/6B1 w - - 0 0"]

1. Bh2 Rg4 2. Kf3 Rd4 3. Bf4 Rc4 4. Ra8 Rc2 5. Ra5 Rc6 6. Be3 Rc3 7. Ke4 Ke7 8. Bd4 Rc2 9. Ra7+ Kd6 10. Be5+ Ke6 11. Ra6+ Kd7 12. Kd5 Re2 13. Ra7+ Kd8 14. Ke6 Rc2 15. Ra8+ Rc8 16. Ra4 Rc6+ 17. Kd5 Rc2 18. Bd6 Rg2 19. Kc6 Ke8 20. Rf4 Rg8 21. Rf3 Kd8 22. Rf2 Ke8 23. Rf1 Rh8 24. Rf3 Rg8 25. Rf6 Rh8 26. Rf1 Rg8 27. Rf2 Kd8 28. Rf7 Ke8 29. Re7+ Kf8 30. Rb7+ Ke8 31. Be5 Rg6+ 32. Kd5 Rh6 33. Rg7 Kd8 34. Bf4 Ra6 35. Bd6 Kc8 36. Rc7+ Kb8 37. Rh7+ Kc8 38. Rg7 Ra5+ 39. Ke6 Rh5 40. Bf4 Rc5 41. Kd6 Rc2 42. Be3 Ra2 43. Kd5 Rc2 44. Bc5 Kd8 45. Bd6 Re2 46. Bf4 Re1 47. Be5 Rd1+ 48. Ke6 Kc8 49. Rc7+ Kd8 50. Rc2 Ra1 51. Bxa1 Ke8 52. Rc8+ 1-0[/pgn]The second game shows a Rook sack at move 50.
Thanks for testing. However, for the second game, it should have been a draw after move 50, but since UCI engines can not claim a draw, they rely on the GUI to claim the draw for them. For some reason the GUI did not do that in this case. Maybe setting the full move counter to 0 confuses the GUI.

I can not reproduce the problem in the first game. Do you have a log file? Already the first move Bf2 gives up the win. On my computer texel finds the best move after 2ms at depth 2 after 112 nodes and 12 TB hits.

Code: Select all

uci
id name Texel 1.06a43 64-bit
id author Peter Osterlund
option name UCI_EngineAbout type string default Texel 1.06a43 64-bit by Peter Osterlund, see http://web.comhem.se/petero2home/javachess/index.html#texel
option name Hash type spin default 16 min 1 max 524288
option name OwnBook type check default false
option name BookFile type string default 
option name Ponder type check default true
option name UCI_AnalyseMode type check default false
option name UCI_Opponent type string default 
option name Strength type spin default 1000 min 0 max 1000
option name Threads type spin default 1 min 1 max 64
option name MultiPV type spin default 1 min 1 max 256
option name UseNullMove type check default true
option name GaviotaTbPath type string default 
option name GaviotaTbCache type spin default 1 min 1 max 2047
option name SyzygyPath type string default 
option name MinProbeDepth type spin default 1 min 0 max 100
option name Clear Hash type button
option name BufferTime type spin default 1000 min 1 max 10000
uciok
setoption name SyzygyPath value /home/petero/chess/rtb/wdl:/home/petero/chess/rtb/dtz
info string Found 145 syzygy tablebases
position fen 8/8/3K2k1/8/8/2n5/8/6BB w - - 0 1
go wtime 10000 winc 100 btime 10000 binc 100
info depth 1
info depth 1 score cp 99 time 1 nodes 3 nps 3000 pv g1f2
info depth 2
info depth 2 score cp 35 upperbound time 2 nodes 100 nps 50000 tbhits 1 pv g1f2
info depth 2 score cp 35 time 2 nodes 101 nps 50500 tbhits 1 pv g1f2
info depth 2 score mate 31 lowerbound time 2 nodes 112 nps 56000 tbhits 12 pv d6e5
info depth 2 score cp 96 time 2 nodes 129 nps 64500 tbhits 13 pv d6e5 g6g5
info depth 2 score mate 45 lowerbound time 2 nodes 131 nps 65500 tbhits 15 pv d6e6
info depth 2 score cp 96 time 2 nodes 135 nps 67500 tbhits 16 pv d6e5 g6g5
info depth 3
info depth 3 score mate 31 lowerbound time 3 nodes 143 nps 47666 tbhits 24 pv d6e5 g6g5
info depth 3 score mate 31 time 3 nodes 228 nps 76000 tbhits 39 pv d6e5 g6f7
info depth 4
info depth 4 score mate 31 time 3 nodes 290 nps 96666 tbhits 47 pv d6e5 g6f7
info depth 5
info depth 5 score mate 31 time 5 nodes 531 nps 106200 tbhits 104 pv d6e5 g6f7 g1c5 f7g6
info depth 6
info depth 6 score mate 31 time 7 nodes 789 nps 112714 tbhits 167 pv d6e5 g6f7 g1c5 f7g6
info depth 7
info depth 7 score mate 31 time 12 nodes 1701 nps 141750 tbhits 330 pv d6e5 g6f7 g1c5 f7g6
info depth 8
info depth 8 score mate 31 time 18 nodes 3224 nps 179111 tbhits 570 pv d6e5 g6f7 g1c5 f7g6
info depth 9
info depth 9 score mate 31 time 39 nodes 8948 nps 229435 tbhits 1537 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7
info depth 10
info depth 10 score mate 31 time 95 nodes 24267 nps 255442 tbhits 4287 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4
info depth 11
info depth 11 score mate 31 time 152 nodes 38719 nps 254730 tbhits 7012 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4
info depth 12
info depth 12 score mate 31 time 352 nodes 98216 nps 279022 tbhits 17328 pv d6e5 g6f7 g1c5 f7g6 h1f3 c3b5 f3e4 g6g5 c5b4 b5a7 b4a5 g5h4 e5d6 a7b5 d6c5 b5a7
info depth 13
info nodes 153863 nps 282317 tbhits 27760 time 545
bestmove d6e5 ponder g6f7
Wow, Little Blitzer is buggy when adjudicating 50 move draw. The missed Win cases are hard to reproduce. I will check later this night when I get back home.
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by syzygy »

hgm wrote:
syzygy wrote:The evaluation?
So you still don't know what this is about.
What a strange conclusion. You don't seem to have understood much of what I wrote...

That you can very precisely explain why it works so poorly doesn't alter the fact that it works poorly.
Yes, I guess you are right and that it works very poorly. That must be why so many engine authors have instead decided to implement your imaginary system.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: My troubles with MultiPV and Syzygy in Stockfish 7

Post by Laskos »

Laskos wrote:Wow, Little Blitzer is buggy when adjudicating 50 move draw. The missed Win cases are hard to reproduce. I will check later this night when I get back home.
Although for Wins I am still getting some draws with Texel (one in 500 games or so) in Cutechess-cli, I observe some odd behavior, the draws seem to cluster and appear at the start of the run. Also, the wrong move is the first move in the game. I guess it's UI-engine communication. Here are 2 games:
[pgn][Event "?"]
[Site "?"]
[Date "2016.02.21"]
[Round "3"]
[White "Texel Syzygy"]
[Black "SF Syzygy"]
[Result "1/2-1/2"]
[PlyCount "43"]
[SetUp "1"]
[TimeControl "10+0.1"]
[FEN "8/8/8/3r4/1k5P/5K2/R7/8 w - - 0 1"]

1. Ra1 {0.00/1 0.42s} 1. ... Kc5 {0.00/1 0.007s} 2. h5 {+0.24/20 0.31s}
2. ... Kd6 {0.00/1 0s} 3. h6 {+0.30/21 0.35s} 3. ... Ke6 {0.00/17 0.20s}
4. h7 {+0.31/20 0.35s} 4. ... Rh5 {0.00/18 0.23s} 5. Ra7 {+0.31/21 0.34s}
5. ... Kf6 {0.00/18 0.086s} 6. Kg4 {+0.24/21 0.32s} 6. ... Rh2 {0.00/21
0.17s} 7. Kg3 {0.00/25 0.27s} 7. ... Rh5 {0.00/20 0.25s} 8. Kg4 {0.00/27
0.31s} 8. ... Rh2 {0.00/55 0.17s} 9. Kg3 {0.00/27 0.27s} 9. ... Rh5 {
0.00/64 0.18s} 10. h8=Q+ {0.00/27 0.30s} 10. ... Rxh8 {0.00/1 0s} 11. Ra1
{0.00/27 0.29s} 11. ... Kg6 {0.00/24 0.19s} 12. Rb1 {0.00/26 0.29s}
12. ... Ra8 {0.00/18 0.23s} 13. Rc1 {0.00/26 0.28s} 13. ... Kf6 {0.00/18
0.21s} 14. Rb1 {0.00/26 0.28s} 14. ... Ke6 {0.00/21 0.21s} 15. Rc1 {
0.00/25 0.23s} 15. ... Kf6 {0.00/33 0.18s} 16. Rb1 {0.00/25 0.27s} 16. ...
Ke6 {0.00/35 0.21s} 17. Rc1 {0.00/25 0.26s} 17. ... Rh8 {0.00/25 0.19s}
18. Ra1 {0.00/25 0.23s} 18. ... Kd5 {0.00/26 0.26s} 19. Rb1 {0.00/25 0.24s
} 19. ... Ke6 {0.00/27 0.19s} 20. Ra1 {0.00/25 0.24s} 20. ... Kd5 {0.00/36
0.22s} 21. Rb1 {0.00/25 0.24s} 21. ... Ke6 {0.00/37 0.20s} 22. Ra1 {
0.00/26 0.24s, Draw by 3-fold repetition} 1/2-1/2

[Event "?"]
[Site "?"]
[Date "2016.02.21"]
[Round "5"]
[White "Texel Syzygy"]
[Black "SF Syzygy"]
[Result "1/2-1/2"]
[PlyCount "188"]
[SetUp "1"]
[TimeControl "10+0.1"]
[FEN "8/8/8/6kr/8/2P5/K1R5/8 w - - 0 1"]

1. Rc1 {0.00/1 0.41s} 1. ... Kf6 {0.00/18 0.26s} 2. Re1 {+0.23/22 0.32s}
2. ... Rh8 {0.00/1 0s} 3. c4 {+0.24/24 0.44s} 3. ... Rh3 {0.00/1 0s} 4.
Kb2 {+0.24/26 0.35s} 4. ... Rd3 {0.00/24 0.15s} 5. Re2 {+0.24/26 0.33s}
5. ... Rg3 {0.00/23 0.053s} 6. Re4 {+0.19/26 0.29s} 6. ... Rd3 {0.00/27
0.18s} 7. Re1 {+0.19/27 0.31s} 7. ... Rg3 {0.00/24 0.090s} 8. c5 {+0.15/26
0.31s} 8. ... Rg5 {0.00/16 0.18s} 9. Rc1 {+0.15/26 0.30s} 9. ... Ke7 {
0.00/20 0.24s} 10. c6 {+0.14/27 0.34s} 10. ... Kd8 {0.00/21 0.19s} 11. Kb3
{+0.14/26 0.31s} 11. ... Kc7 {0.00/19 0.17s} 12. Kb4 {0.00/26 0.28s}
12. ... Rg7 {0.00/24 0.20s} 13. Rc4 {0.00/27 0.34s} 13. ... Rg2 {0.00/25
0.18s} 14. Ka4 {0.00/27 0.23s} 14. ... Rg5 {0.00/26 0.19s} 15. Rc2 {
0.00/28 0.30s} 15. ... Rg4+ {0.00/29 0.21s} 16. Ka5 {0.00/29 0.38s}
16. ... Rg1 {0.00/33 0.20s} 17. Rc3 {0.00/27 0.23s} 17. ... Rd1 {0.00/29
0.27s} 18. Kb4 {0.00/27 0.24s} 18. ... Rb1+ {0.00/33 0.19s} 19. Kc5 {
0.00/28 0.22s} 19. ... Ra1 {0.00/31 0.20s} 20. Rc2 {0.00/26 0.23s} 20. ...
Rg1 {0.00/30 0.27s} 21. Rc4 {0.00/27 0.25s} 21. ... Rb1 {0.00/30 0.26s}
22. Rc3 {0.00/28 0.29s} 22. ... Ra1 {0.00/48 0.22s} 23. Rc4 {0.00/27 0.23s
} 23. ... Rb1 {0.00/41 0.20s} 24. Rc2 {0.00/28 0.24s} 24. ... Ra1 {0.00/33
0.21s} 25. Kb5 {0.00/27 0.22s} 25. ... Rb1+ {0.00/39 0.21s} 26. Kc5 {
0.00/29 0.21s} 26. ... Ra1 {0.00/45 0.21s} 27. Rc3 {0.00/28 0.21s} 27. ...
Rh1 {0.00/33 0.33s} 28. Kb4 {0.00/27 0.25s} 28. ... Rb1+ {0.00/50 0.24s}
29. Ka3 {0.00/30 0.20s} 29. ... Ra1+ {0.00/27 0.24s} 30. Kb3 {0.00/30
0.19s} 30. ... Rb1+ {0.00/36 0.21s} 31. Ka2 {0.00/29 0.27s} 31. ... Rd1 {
0.00/38 0.21s} 32. Kb3 {0.00/28 0.21s} 32. ... Rb1+ {0.00/45 0.065s} 33.
Ka2 {0.00/30 0.30s} 33. ... Rd1 {0.00/56 0.24s} 34. Rc2 {0.00/28 0.18s}
34. ... Rg1 {0.00/25 0.27s} 35. Ka3 {0.00/27 0.19s} 35. ... Ra1+ {0.00/25
0.52s} 36. Kb3 {0.00/30 0.18s} 36. ... Rb1+ {0.00/22 0.25s} 37. Ka3 {
0.00/29 0.17s} 37. ... Ra1+ {0.00/56 0.23s} 38. Kb2 {0.00/29 0.17s}
38. ... Rd1 {0.00/26 0.23s} 39. Rc4 {0.00/28 0.18s} 39. ... Re1 {0.00/25
0.29s} 40. Rc2 {0.00/27 0.17s} 40. ... Rd1 {0.00/58 0.20s} 41. Rc4 {
0.00/29 0.38s} 41. ... Re1 {0.00/54 0.20s} 42. Rc3 {0.00/29 0.24s} 42. ...
Rd1 {0.00/27 0.25s} 43. Rc5 {0.00/28 0.17s} 43. ... Rf1 {0.00/27 0.20s}
44. Kb3 {0.00/27 0.17s} 44. ... Rf3+ {0.00/27 0.22s} 45. Kb4 {+0.01/26
0.16s} 45. ... Rf1 {0.00/34 0.25s} 46. Rc4 {0.00/27 0.15s} 46. ... Rb1+ {
0.00/34 0.085s} 47. Kc5 {0.00/29 0.15s} 47. ... Rg1 {0.00/34 0.35s} 48.
Kb5 {0.00/27 0.18s} 48. ... Rb1+ {0.00/34 0.049s} 49. Kc5 {0.00/28 0.13s}
49. ... Rg1 {0.00/43 0.18s} 50. Kb5 {0.00/28 0.19s} 50. ... Ra1 {0.00/31
0.49s} 51. Kb4 {0.00/27 0.16s} 51. ... Rb1+ {0.00/49 0.17s} 52. Ka3 {
0.00/28 0.14s} 52. ... Ra1+ {0.00/31 0.22s} 53. Kb2 {0.00/29 0.14s}
53. ... Rd1 {0.00/28 0.25s} 54. Rc2 {0.00/27 0.14s} 54. ... Rd5 {0.00/36
0.16s} 55. Rc1 {0.00/27 0.14s} 55. ... Rf5 {0.00/25 0.17s} 56. Ra1 {
0.00/26 0.13s} 56. ... Kxc6 {0.00/23 0.16s} 57. Rc1+ {0.00/25 0.13s}
57. ... Kb5 {0.00/22 0.23s} 58. Ra1 {0.00/26 0.15s} 58. ... Rh5 {0.00/23
0.19s} 59. Rc1 {0.00/25 0.13s} 59. ... Kb4 {0.00/22 0.15s} 60. Rc6 {
0.00/26 0.13s} 60. ... Rg5 {0.00/22 0.15s} 61. Rc1 {0.00/26 0.13s} 61. ...
Rg2+ {0.00/28 0.17s} 62. Rc2 {0.00/28 0.15s} 62. ... Rg6 {0.00/30 0.17s}
63. Ka1 {0.00/26 0.13s} 63. ... Kb3 {0.00/24 0.14s} 64. Kb1 {0.00/26 0.12s
} 64. ... Rg1+ {0.00/27 0.16s} 65. Rc1 {0.00/17 0s} 65. ... Rg2 {0.00/32
0.16s} 66. Rf1 {0.00/26 0.13s} 66. ... Rb2+ {0.00/25 0.16s} 67. Kc1 {
0.00/26 0.11s} 67. ... Rc2+ {0.00/33 0.16s} 68. Kd1 {0.00/27 0.13s}
68. ... Rc8 {0.00/19 0.15s} 69. Rf3+ {0.00/25 0.13s} 69. ... Kb2 {0.00/20
0.19s} 70. Re3 {0.00/26 0.13s} 70. ... Rd8+ {0.00/20 0.17s} 71. Ke1 {
0.00/27 0.13s} 71. ... Kc2 {0.00/22 0.17s} 72. Ra3 {0.00/25 0.12s} 72. ...
Rc8 {0.00/23 0.15s} 73. Ra2+ {0.00/25 0.12s} 73. ... Kb3 {0.00/23 0.16s}
74. Ra1 {0.00/25 0.13s} 74. ... Rc7 {0.00/21 0.16s} 75. Rd1 {0.00/25 0.13s
} 75. ... Kc2 {0.00/24 0.13s} 76. Rd2+ {0.00/25 0.12s} 76. ... Kc3 {
0.00/25 0.071s} 77. Ra2 {0.00/26 0.12s} 77. ... Re7+ {0.00/22 0.14s} 78.
Kd1 {0.00/25 0.10s} 78. ... Re5 {0.00/22 0.13s} 79. Re2 {0.00/26 0.13s}
79. ... Rh5 {0.00/22 0.13s} 80. Rf2 {0.00/25 0.12s} 80. ... Rh1+ {0.00/22
0.15s} 81. Ke2 {0.00/27 0.12s} 81. ... Rh5 {0.00/24 0.042s} 82. Rf1 {
0.00/25 0.12s} 82. ... Kb2 {0.00/23 0.16s} 83. Rd1 {0.00/25 0.12s} 83. ...
Rh8 {0.00/18 0.18s} 84. Rg1 {0.00/25 0.12s} 84. ... Kc2 {0.00/17 0.13s}
85. Ra1 {0.00/25 0.13s} 85. ... Rh7 {0.00/24 0.14s} 86. Ra2+ {0.00/25
0.12s} 86. ... Kb3 {0.00/23 0.18s} 87. Ra1 {0.00/25 0.12s} 87. ... Re7+ {
0.00/28 0.16s} 88. Kd2 {0.00/25 0.11s} 88. ... Rg7 {0.00/24 0.15s} 89.
Rb1+ {0.00/24 0.11s} 89. ... Kc4 {0.00/27 0.13s} 90. Ra1 {0.00/25 0.12s}
90. ... Rg2+ {0.00/20 0.11s} 91. Ke3 {0.00/26 0.10s} 91. ... Rg3+ {0.00/20
0.14s} 92. Kd2 {0.00/25 0.12s} 92. ... Rg2+ {0.00/36 0.12s} 93. Ke3 {
0.00/26 0.11s} 93. ... Rg3+ {0.00/39 0.13s} 94. Kd2 {0.00/25 0.11s}
94. ... Rg2+ {0.00/42 0.13s, Draw by 3-fold repetition} 1/2-1/2[/pgn]