Ah, thanks ... that makes sense. With that I've got a complete implementation of probe_table in pythonchess. I'll look into probe_ab for suicide chess next.
As a side effect atomic tablebase probing code is complete. I also ported it to Daniel Dugovic's multivariant Stockfish fork. Here's the complete 6 piece set including stats and md5 sums for comparison: https://syzygytables.info/atomic/
Suicide chess tablebases (stalemated player wins)
Moderators: hgm, Rebel, chrisw

 Posts: 42
 Joined: Sat May 16, 2015 11:41 pm

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
Excellent!niklasf wrote:As a side effect atomic tablebase probing code is complete. I also ported it to Daniel Dugovic's multivariant Stockfish fork. Here's the complete 6 piece set including stats and md5 sums for comparison: https://syzygytables.info/atomic/
Interestingly, the md5sums don't match in 5 cases:
 KPvK.atbw
 KQvK.atbz
 KPPvKPP.atbw
 KRPvKBP.atbw
 KRPvKBP.atbz
The first two almost certainly have to do with this fix, which also changed the md5sums for KPvK, KQvK, KRvK (.rbtw and .rtbz). Your KPvK.atbw and KQvK.atbz files are indeed a bit smaller than mine (7248 and 3536 bytes versus 7632 and 4112 bytes).
The mismatch in KPPvKPP is a problem in my table. My stats are this:
Code: Select all
########## KPPvKPP ##########
White to move:
3773131244 positions win in 1 ply.
91164 positions win in 2 ply.
339849016 positions win in 3 ply.
220 positions win in 4 ply.
...
283556 positions lose in 0 ply.
400288 positions lose in 1 ply.
2498780648 positions lose in 2 ply.
51292 positions lose in 3 ply.
Black to move:
3773131248 positions win in 1 ply.
91164 positions win in 2 ply.
339849020 positions win in 3 ply.
220 positions win in 4 ply.
...
283556 positions lose in 0 ply.
400288 positions lose in 1 ply.
2498780644 positions lose in 2 ply.
51292 positions lose in 3 ply.
...
Since KPPvKPP is symmetric and has pawns my verification program cannot handle it (complicated explanation omitted). I may have fixed a bug at some point that only affected KPPvKPP, but I don't remember now.
Our KRPvKBP statistics have a number of differences for white to move (and win). I have:
Code: Select all
########## KRPvKBP ##########
White to move:
3892382286 positions win in 1 ply.
246837 positions win in 2 ply.
860178675 positions win in 3 ply.
76240 positions win in 4 ply.
307725146 positions win in 5 ply.
56791 positions win in 6 ply.
...
5418429732 positions are wins.
1361 positions are cursed wins.
6651524926 positions are draws.
1340134 positions are cursed losses.
80702422 positions are losses.
Code: Select all
########## KRPvKBP ##########
White to move:
3892382244 positions win in 1 ply.
246837 positions win in 2 ply.
860178675 positions win in 3 ply.
76240 positions win in 4 ply.
307725187 positions win in 5 ply.
56791 positions win in 6 ply.
...
5418429731 positions are wins.
1361 positions are cursed wins.
6651524927 positions are draws.
1340134 positions are cursed losses.
80702422 positions are losses.
I will regenerate these five tables and I expect that they will then be identical to yours. I will then see if I can find out what bug I may have fixed. Almost certainly something having to do with en passant.

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
I regenerated the four tables and now everything is in agreement.syzygy wrote:I will regenerate these five tables and I expect that they will then be identical to yours. I will then see if I can find out what bug I may have fixed. Almost certainly something having to do with en passant.
I also verified my old KRPvKBP table and indeed some inconsistencies were detected.

 Posts: 42
 Joined: Sat May 16, 2015 11:41 pm
Re: Suicide chess tablebases (stalemated player wins)
While the 6 piece tables are still beeing generated (as you said this will take a long time) I've been making some slow progress on the probing code.
For the moment I am testing with 5 piece tables. I have uploaded them here, including checksums: https://syzygytables.info/suicide/
The probe_wdl / probe_ab implementation seems to work nicely.
I am having trouble with the DTZ in this position:
[d] 3q4/8/3b4/8/8/8/3N4/1K1Q4 b   0 1
I believe the DTZ should be 4, but my probing code in pythonchess gives it as DTZ 1 directly after doing probe_dtz_table https://github.com/niklasf/pythonchess ... y.py#L1753. idx is 32881184. enc_type is 0 (with more than 3 unique pieces).
I am now trying to think of the most effective way to debug this. If you prefer talking about C/C++ code, perhaps I should instead try to implement it in Stockfish first (although with the C++ rewrite it has diverged quite a bit from your original probing code) ... Any hints or things I could try?
For the moment I am testing with 5 piece tables. I have uploaded them here, including checksums: https://syzygytables.info/suicide/
The probe_wdl / probe_ab implementation seems to work nicely.
I am having trouble with the DTZ in this position:
[d] 3q4/8/3b4/8/8/8/3N4/1K1Q4 b   0 1
I believe the DTZ should be 4, but my probing code in pythonchess gives it as DTZ 1 directly after doing probe_dtz_table https://github.com/niklasf/pythonchess ... y.py#L1753. idx is 32881184. enc_type is 0 (with more than 3 unique pieces).
I am now trying to think of the most effective way to debug this. If you prefer talking about C/C++ code, perhaps I should instead try to implement it in Stockfish first (although with the C++ rewrite it has diverged quite a bit from your original probing code) ... Any hints or things I could try?

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
Oh wow, you are generating them!niklasf wrote:While the 6 piece tables are still beeing generated (as you said this will take a long time) I've been making some slow progress on the probing code.
Are you sure you want to generate the .stbw/z tables (FICS rules) and not the .gtbw/z tables (international rules)?For the moment I am testing with 5 piece tables. I have uploaded them here, including checksums: https://syzygytables.info/suicide/
Of course the .stbw/z tables have the advantage that I can check the checksums. And your 2345 pieces tables all have identical md5sums to mine. Of course that only means there are no hardware or compiler problems and that I did not accidentally change the code after I generated my set of tables.
I should be able to understand Python code.The probe_wdl / probe_ab implementation seems to work nicely.
I am having trouble with the DTZ in this position:
[d] 3q4/8/3b4/8/8/8/3N4/1K1Q4 b   0 1
I believe the DTZ should be 4, but my probing code in pythonchess gives it as DTZ 1 directly after doing probe_dtz_table https://github.com/niklasf/pythonchess ... y.py#L1753. idx is 32881184. enc_type is 0 (with more than 3 unique pieces).
I am now trying to think of the most effective way to debug this. If you prefer talking about C/C++ code, perhaps I should instead try to implement it in Stockfish first (although with the C++ rewrite it has diverged quite a bit from your original probing code) ... Any hints or things I could try?
So you found the position starting from the longest black win (5 ply):
[d]3q4/8/3b4/8/8/8/3N4/2KQ4 w  
The .stbz file holds white to move here (I guess), so one of black's moves should lead to a black win in 4 ply.
OK, it seems I did implement a probe utility after all as part of a proofnumber search solver. Unfortunately it also returns dtz = 2 for the 5ply position and +1 for the 4ply position.
So this needs some investigation...

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
Strange, for other tables things seem to be fine (apart from the known offbyone in certain cases).
I don't see what could be special about the particular position or table that is giving problems. But with some effort I will be able to track it down.
Please let me know if you find other problems. Hopefully at least the WDL tables are all correct. And hopefully the DTZ problem is limited to just a few tables...
I don't see what could be special about the particular position or table that is giving problems. But with some effort I will be able to track it down.
Please let me know if you find other problems. Hopefully at least the WDL tables are all correct. And hopefully the DTZ problem is limited to just a few tables...

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
It seems KQBvQB has similar problems.
[d]3q4/8/3b4/8/8/3B4/8/1K1Q4 w  
White wins in 4 ply, but probe_dtz() returns 2.
[d]8/8/8/8/7q/1B4b1/Q1K5/8 b  
Black wins in 4 ply, but probe_dtz() returns 1.
Another one:
[d]R7/R7/PR6/8/8/8/8/4b3 w  
White wins in 4 ply, but probe_dtz() returns 2.
But this one works fine:
[d]8/2P5/8/2R1P3/P7/7q/8/8 w  
White wins in 4 ply, probe_dtz() returns 4.
[d]3q4/8/3b4/8/8/3B4/8/1K1Q4 w  
White wins in 4 ply, but probe_dtz() returns 2.
[d]8/8/8/8/7q/1B4b1/Q1K5/8 b  
Black wins in 4 ply, but probe_dtz() returns 1.
Another one:
[d]R7/R7/PR6/8/8/8/8/4b3 w  
White wins in 4 ply, but probe_dtz() returns 2.
But this one works fine:
[d]8/2P5/8/2R1P3/P7/7q/8/8 w  
White wins in 4 ply, probe_dtz() returns 4.

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
This is strange:
[d]8/4R1Q1/5N2/8/q7/8/8/2k5 b  
The generator outputs this is as the longest win for white, 4 ply.
But Qg4 wins for black (and probe_wdl/dtz reflect this correctly).
[d]2R5/N1Q5/8/8/8/3k4/8/3q4 b  
Supposed to be a black win in 5 ply, but for example Qc2 Qxc2 Kxc2 Rxc2 wins in 2 ply (DTZ).
This suggests the problem might be in the reporting of the longest wins (in certain cases with low max DTZ).
However, this hypothesis fails for the original problem position:
[d]3q4/8/3b4/8/8/8/3N4/1K1Q4 b   0 1
Black can win in 4 ply (DTZ) with Qc7 Ka1 Qc4 Nxc4 Ba3 Nxa3, for example.
Black certainly does not have a win in 1 or 2 ply.
[d]8/4R1Q1/5N2/8/q7/8/8/2k5 b  
The generator outputs this is as the longest win for white, 4 ply.
But Qg4 wins for black (and probe_wdl/dtz reflect this correctly).
[d]2R5/N1Q5/8/8/8/3k4/8/3q4 b  
Supposed to be a black win in 5 ply, but for example Qc2 Qxc2 Kxc2 Rxc2 wins in 2 ply (DTZ).
This suggests the problem might be in the reporting of the longest wins (in certain cases with low max DTZ).
However, this hypothesis fails for the original problem position:
[d]3q4/8/3b4/8/8/8/3N4/1K1Q4 b   0 1
Black can win in 4 ply (DTZ) with Qc7 Ka1 Qc4 Nxc4 Ba3 Nxa3, for example.
Black certainly does not have a win in 1 or 2 ply.

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
Something trivial is going wrong with QRNvKQ:
The 4ply win for white is with white to move. The 5ply win for black is with black to move. This probably has to do with switching of colours. The positions might be right, but the piece types and colours may have to be permuted.
But this does not explain the original problem position.
Let's see:
This makes sense. But something goes wrong...
Code: Select all
########## QRNvKQ ##########
White to move:
35207378 positions win in 1 ply.
25146357 positions win in 2 ply.
12 positions win in 4 ply.
60353747 positions are wins.
2504186 positions are draws.
51511427 positions are losses.
51507632 positions lose in 1 ply.
1877 positions lose in 2 ply.
1911 positions lose in 3 ply.
7 positions lose in 5 ply.
Black to move:
47021881 positions win in 1 ply.
21747453 positions win in 2 ply.
2 positions win in 3 ply.
27 positions win in 4 ply.
68769363 positions are wins.
2450555 positions are draws.
43149442 positions are losses.
43148147 positions lose in 1 ply.
1295 positions lose in 3 ply.
Longest win for white: 4 ply; 8/4R1Q1/5N2/8/q7/8/8/2k5 b  
Longest win for black: 5 ply; 2R5/N1Q5/8/8/8/3k4/8/3q4 b  
But this does not explain the original problem position.
Let's see:
Code: Select all
########## KQNvQB ##########
White to move:
31917632 positions win in 1 ply.
33281532 positions win in 2 ply.
95 positions win in 4 ply.
65199259 positions are wins.
1329566 positions are draws.
47840535 positions are losses.
47827348 positions lose in 1 ply.
8847 positions lose in 2 ply.
4316 positions lose in 3 ply.
8 positions lose in 4 ply.
16 positions lose in 5 ply.
Black to move:
47774166 positions win in 1 ply.
19314330 positions win in 2 ply.
10 positions win in 3 ply.
27 positions win in 4 ply.
67088533 positions are wins.
914916 positions are draws.
46365911 positions are losses.
46365400 positions lose in 1 ply.
511 positions lose in 3 ply.
Longest win for white: 4 ply; 8/8/8/8/8/6bq/1N6/QK6 w  
Longest win for black: 5 ply; 3q4/8/3b4/8/8/8/3N4/2KQ4 w  

 Posts: 5584
 Joined: Tue Feb 28, 2012 11:56 pm
Re: Suicide chess tablebases (stalemated player wins)
Apparently I have already fixed this. (I probably fixed it between generating 5piece and 6piece tables, but I don't remember exactly.)syzygy wrote:Something trivial is going wrong with QRNvKQ:
You probably have this:
Code: Select all
########## QRNvKQ ##########
White to move:
35207378 positions win in 1 ply.
25146357 positions win in 2 ply.
12 positions win in 4 ply.
60353747 positions are wins.
2504186 positions are draws.
51511427 positions are losses.
51507632 positions lose in 1 ply.
1877 positions lose in 2 ply.
1911 positions lose in 3 ply.
7 positions lose in 5 ply.
Black to move:
47021881 positions win in 1 ply.
21747453 positions win in 2 ply.
2 positions win in 3 ply.
27 positions win in 4 ply.
68769363 positions are wins.
2450555 positions are draws.
43149442 positions are losses.
43148147 positions lose in 1 ply.
1295 positions lose in 3 ply.
Longest win for white: 4 ply; 8/4R1Q1/5N2/8/q7/8/8/2k5 w  
Longest win for black: 5 ply; 2R5/N1Q5/8/8/8/3k4/8/3q4 w  