Syzygy EGTB's via Torrent Thread

[Ozymandias]

Depends on what you compare it with, my stats for today are more than 10G in less than 9 hours. Others have even more.

[jshriver]

Thanks to everyone who is participating. I see we are pushing 4 megs now, WOW. Plus we have 4 seeders for WDL, 9 for 3-4-5 (plus webseed)! Great job.

-Josh

[RJN]

I have finished WDL, and ran a checksum: all clean, and it should be a fast seeder.

As always, everyone should ALWAYS confirm on their end that they have no corruption once they finish a torrent.

[IQ]

I also completed the WDL torrent. Checked everything against the original posters md5 and the "run.pl --verify" of the syzygy generator package. Everything is fine! Also one of the new Stockfish4 commits changed the iterator increments. I had to change a couple of "i++" to "++i" and "ptr++" to "++ptr" to compile tbprobe.cpp. Seems to work pretty good.

I copied the 6 piece WDL over to the 5 piece WDL and DTZ. Stockfish plays fine but sometimes if a 6 piece postiion arises on the board stockfish faults with an exception. I guess, if you want to use the 6 piece WDL you also need the 6 piece DTZ for Stockfish to work.

[syzygy]

I also completed the WDL torrent. Checked everything against the original posters md5 and the "run.pl --verify" of the syzygy generator package.

It is not really necessary anymore to do a "run.pl --verify" on them.
If you have the generator binaries, I recommend running tbcheck as follows:

Code: Select all

$ tbcheck *.rtbw *.rtbz

This recalculates a checksum for each file and compares it with a checksum embedded in the file. Joshua has already verified (using tbcheck --compare) that the embedded checksums have the correct values. If any bit was flipped during the download, this test should catch it.

I am afraid that *.rtbw will not expand correctly on Windows, which makes this method a bit cumbersome. On Windows it therefore seems best to use md5sum.

Everything is fine! Also one of the new Stockfish4 commits changed the iterator increments. I had to change a couple of "i++" to "++i" and "ptr++" to "++ptr" to compile tbprobe.cpp. Seems to work pretty good.

I copied the 6 piece WDL over to the 5 piece WDL and DTZ. Stockfish plays fine but sometimes if a 6 piece postiion arises on the board stockfish faults with an exception. I guess, if you want to use the 6 piece WDL you also need the 6 piece DTZ for Stockfish to work.

I am assuming you copied the interface code over to the current Stockfish sources. Instead, try this one:
https://github.com/syzygy1/Stockfish
It should compile without problems (ignore the few warnings).

(In case you already compiled from these sources then something seems wrong since Stockfish should not crash. I'll have a look later to see if I need to fix anything.)

[IQ]

I already used the new interface code. Here are some of the games where Stockfish caused an exception/provided no move when a 6 piece position arose on the board (tbhits during search worked fine). Remember that i have a WDL directory with all 3-4-5-6 pieces and an DTZ directory with only 3-4-5 pieces. Both are pointed to by the new Syzygy Path option:

[Date "2013.09.16"]
[White "Stockfish"]
[Black "Rybka 4 x64"]
[Result "0-1"]
[PlyCount "152"]
[EventDate "2013.09.16"]

1. d4 d5 2. c4 c6 3. Nc3 Nf6 4. Nf3 e6 5. e3 Nbd7 6. Qc2 Bd6 7. Bd3 O-O 8. O-O
dxc4 9. Bxc4 a6 10. Rd1 b5 11. Be2 Qc7 12. a3 c5 13. dxc5 Nxc5 14. Bxb5 Bb7 15.
Bc4 Bxf3 16. gxf3 Bxh2+ 17. Kg2 Be5 18. Bd2 Qb7 19. b4 Ncd7 20. Rac1 Nb6 21.
Be2 Rfc8 22. Qb3 h6 23. Na4 Nbd7 24. Kg1 Bd6 25. Qd3 Rc6 26. f4 Be7 27. Bc3 Nd5
28. Bd4 Bf6 29. f5 Bxd4 30. Qxd4 Rac8 31. Rxc6 Qxc6 32. Nc5 Nxc5 33. Rc1 Qb6
34. bxc5 Qc6 35. Bf3 a5 36. Kh1 g6 37. fxe6 fxe6 38. Be4 Kf7 39. Qe5 g5 40. Bf3
Rf8 41. Rb1 Qd7 42. Rd1 Kg6 43. Be4+ Kf7 44. f4 Kg8 45. fxg5 h5 46. c6 Qf7 47.
g6 Qf6 48. Qxh5 Qh8 49. Qxh8+ Kxh8 50. Bxd5 exd5 51. Rxd5 Kg7 52. Kg2 Kxg6 53.
c7 Rc8 54. Rxa5 Rxc7 55. a4 Kf6 56. Kf3 Ke6 57. Rh5 Rf7+ 58. Ke4 Kd7 59. a5 Kc6
60. a6 Kb6 61. Rh6+ Kc5 62. Rh1 Re7+ 63. Kf4 Rf7+ 64. Kg5 Ra7 65. Ra1 Kd5 66.
Kf6 Ra8 67. a7 Rf8+ 68. Ke7 Ra8 69. Ra4 Ke5 70. Kd7 Kf6 71. Ra5 Kg6 72. e4 Kg7
73. e5 Kf8 74. Ra3 Kg7 75. e6 Kf6 76. e7 Kf5 Stockfish caused an exception 0-1

[Date "2013.09.16"]
[Round "?"]
[White "Stockfish"]
[Black "Houdini 3 Pro x64"]
[Result "*"]
[PlyCount "162"]
[EventDate "2013.09.16"]

1. d4 Nf6 2. c4 e6 3. Nf3 d5 4. Nc3 c6 5. e3 Nbd7 6. Qc2 Bd6 7. Bd3 O-O 8. O-O
dxc4 9. Bxc4 a6 10. Rd1 b5 11. Be2 Qc7 12. a3 c5 13. dxc5 Bxc5 14. b4 Be7 15.
Bb2 Bb7 16. Rac1 Rac8 17. a4 bxa4 18. Qxa4 Qb8 19. Na2 Rfd8 20. h3 h6 21. Bd4
Rxc1 22. Nxc1 e5 23. Bb2 Qc7 24. Qa5 Qxa5 25. bxa5 e4 26. Nd4 Rc8 27. Nf5 Bf8
28. Nb3 Rc2 29. Rd2 Rxd2 30. Nxd2 g6 31. Nd4 Nc5 32. Ba3 Nfd7 33. f3 Nf6 34.
Bf1 h5 35. Be2 Kg7 36. N4b3 exf3 37. gxf3 Nfd7 38. h4 Be7 39. Nxc5 Bxc5 40.
Bxc5 Nxc5 41. f4 Kf6 42. Bc4 Ke7 43. Kf2 f5 44. Nb3 Ne4+ 45. Ke1 Bc8 46. Nd4
Bb7 47. Kd1 Nd6 48. Be2 Kf6 49. Kc2 Ne4 50. Nb3 Ke6 51. Nd2 Nc5 52. Kc3 Ke7 53.
Nf3 Ke6 54. Ne5 g5 55. Kb4 gxf4 56. exf4 Ne4 57. Bxh5 Kf6 58. Be2 Bc8 59. Kc4
Kg7 60. Kd4 Kf6 61. Bf3 Nd2 62. Bd5 Nf1 63. h5 Kg7 64. Nc4 Bd7 65. Bb7 Bb5 66.
Nd6 Be2 67. Nxf5+ Kh7 68. Kc3 Nh2 69. Nd4 Bf1 70. Ne6 Ng4 71. Nc5 Nf6 72. Bxa6
Bg2 73. Bd3+ Kh6 74. a6 Nd5+ 75. Kd4 Nb4 76. Be4 Nxa6 77. Bxg2 Nc7 78. Bc6 Kg7
79. Ke5 Kh6 80. Kd6 Kg7 81. Kxc7 Kh6 * Stockish delivered no move

[Date "2013.09.16"]
[White "Houdini 3 Pro x64"]
[Black "Stockfish"]
[Result "1-0"]
[ECO "B92"]
[PlyCount "155"]
[EventDate "2013.09.16"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be3 e5 7. Nb3 Be6 8. Be2
Be7 9. O-O O-O 10. Qd3 Nbd7 11. Nd5 Bxd5 12. exd5 Nc5 13. Nxc5 dxc5 14. Bf3 Re8
15. Rae1 Bd6 16. Bg5 h6 17. Bh4 b5 18. b3 b4 19. Re3 a5 20. h3 a4 21. Rfe1 g6
22. R3e2 Kg7 23. Qc4 Qb6 24. Re3 axb3 25. axb3 Qa6 26. Bxf6+ Kxf6 27. Qe4 Kg7
28. Be2 Qb6 29. g4 Qd8 30. Bd3 Qg5 31. Rg3 Ra2 32. Kh2 Rea8 33. Kh1 h5 34. Kg2
Ra1 35. Rxa1 Rxa1 36. Qe3 Qh4 37. Be4 Ra8 38. Qe2 hxg4 39. Rxg4 Qf6 40. Qb5 Rf8
41. Qa6 Rh8 42. Rg3 Qe7 43. Kf1 Rh4 44. Qd3 Kh8 45. Rg1 Qf8 46. Qe3 Kh7 47. Rg4
Rh5 48. Rg5 Rh6 49. Qf3 f5 50. Bxf5 Qf6 51. Qg4 Be7 52. Rxg6 Rxg6 53. Qh5+ Kg7
54. Bxg6 Qxg6 55. Qxe5+ Kf7 56. f3 Qf6 57. Qe4 Ke8 58. f4 Kd7 59. Ke2 Bd6 60.
Kf3 Qh4 61. Qf5+ Kc7 62. Qf7+ Kb6 63. Qf5 c4 64. bxc4 Kc5 65. Qc8+ Kd4 66. Qf5
Kxc4 67. Qe4+ Kc5 68. Qf5 Qh8 69. Kg4 Qg7+ 70. Kf3 Qc3+ 71. Kg4 Qd2 72. h4 Qg2+
73. Kh5 Qf3+ 74. Kg6 Qxf4 75. Qxf4 Bxf4 76. Kf5 Bg3 77. Kg4 Bxh4 78. d6 Stockfish caused an exception 1-0

I also completed the WDL torrent. Checked everything against the original posters md5 and the "run.pl --verify" of the syzygy generator package.

It is not really necessary anymore to do a "run.pl --verify" on them.
If you have the generator binaries, I recommend running tbcheck as follows:

Code: Select all

$ tbcheck *.rtbw *.rtbz

This recalculates a checksum for each file and compares it with a checksum embedded in the file. Joshua has already verified (using tbcheck --compare) that the embedded checksums have the correct values. If any bit was flipped during the download, this test should catch it.

I am afraid that *.rtbw will not expand correctly on Windows, which makes this method a bit cumbersome. On Windows it therefore seems best to use md5sum.

Everything is fine! Also one of the new Stockfish4 commits changed the iterator increments. I had to change a couple of "i++" to "++i" and "ptr++" to "++ptr" to compile tbprobe.cpp. Seems to work pretty good.

I copied the 6 piece WDL over to the 5 piece WDL and DTZ. Stockfish plays fine but sometimes if a 6 piece postiion arises on the board stockfish faults with an exception. I guess, if you want to use the 6 piece WDL you also need the 6 piece DTZ for Stockfish to work.

I am assuming you copied the interface code over to the current Stockfish sources. Instead, try this one:
https://github.com/syzygy1/Stockfish
It should compile without problems (ignore the few warnings).

(In case you already compiled from these sources then something seems wrong since Stockfish should not crash. I'll have a look later to see if I need to fix anything.)

[IQ]

Addendum to my previous post: stockfish does not really throw an exception. The fritz interface was misleading there. The endgame probing code in uci mode exits gracefully with f.e. "Could not open KRNPvKP for reading". If this is intended it just means one cannot use the WDL 6 piece bases without the DTZ 6 piece bases.

Until the 6 piece DTZ torrent is completed i just patched the probing code to only access the DTZ for 5 pieces or less. Now everything seems to work.

[syzygy]

Addendum to my previous post: stockfish does not really throw an exception. The fritz interface was misleading there. The endgame probing code in uci mode exits gracefully with f.e. "Could not open KRNPvKP for reading". If this is intended it just means one cannot use the WDL 6 piece bases without the DTZ 6 piece bases.

This was intended, but it is not very nice.

I'll fix it by using the WDL tables to filter out bad moves at the root when DTZ tables are not available. The engine is then not guaranteed to make progress, but in practice it should rarely fail to win positions that are won. In theory one could then even delete the "trivial" DTZ tables, i.e. those with a sufficiently small max DTZ.

Good to know that Stockfish did not throw the exception, so that I don't have to look for what could be causing that.

[velmarin]

I compiled Stockfish with Intel compiler, with a few changes, (__builtin_bswap64)ect.
I'm testing with parts 3,4,5, for now, all perfect.
Displays TBhits and good performance with a minute games.
There are no exceptions.
I tester after with 6 pieces.
If all goes well will try to test my engines.


Thank you.

[IQ]

I compiled Stockfish with Intel compiler, with a few changes, (__builtin_bswap64)ect.
I'm testing with parts 3,4,5, for now, all perfect.
Displays TBhits and good performance with a minute games.
There are no exceptions.
I tester after with 6 pieces.
If all goes well will try to test my engines.

There are no problems and no crashes with the tablebase code added. My issue above was a fringe case: it occurs only if have the 6 piece wdl bases WITHOUT the corresponding 6 piece DTZ bases. So normally no problems at all - i already played hundreds of games. And even that fringe case is easily solved if you limit dtz probing at the root to a maximum of 5 pieces. Theoretically in some situations this means that stockfish might not know how to make progress, but so far it always found the winning line through search + WDL 6 piece probing. This will do for me until the DTZ torrent is finished.