7-men Syzygy attempt

[Dann Corbit]

KRBNvKQ.rtbw should be fine.
Unfortunately, KRBNvKQ.rtbz will have to be regenerated (-z).

I have now tested the change and it seems to work well.

The 7 man branch says the last change was 18 days ago.
Does the master branch do 7 man files now?

[syzygy]

KRBNvKQ.rtbw should be fine.
Unfortunately, KRBNvKQ.rtbz will have to be regenerated (-z).

I have now tested the change and it seems to work well.

The 7 man branch says the last change was 18 days ago.
Does the master branch do 7 man files now?

Not yet.

[noobpwnftw]

Regenerating KRBNvKQN.
The other tables shouldn't be affected by the look of it, like KBBBNvKQ which had a max of 268 ply generated by the refactor branch, KRBBNvKQ had 380 ply by master branch.
Please help me with checking.

[syzygy]

Regenerating KRBNvKQN.
The other tables shouldn't be affected by the look of it, like KBBBNvKQ which had a max of 268 ply generated by the refactor branch, KRBBNvKQ had 380 ply by master branch.
Please help me with checking.

Those are indeed not affected. 380 ply is stored as (380-100)/2-1 = 139, which fits easily in 8 bits.

The compressed format itself can deal with values up to 4095 (the first stage of the compressor in fact extends the starting set of values with symbols representing longer strings of values, resulting in up to 4095 values/symbols which are then compressed using a huffman code). The problem here was in an array that remaps the stored values. Since there are separate value ranges for win, cursed win, loss, cursed loss (i.e. the same value can mean different things depending on the type of position according to the WDL table), the compressor first sorts each range by frequency and maps the most frequent value of each range to the same symbol, etc. When decompressing, this mapping has to be undone with an array lookup. The entries of that array should of course hold the whole value and not just the lowest 8 bits if there are values > 255... For 6 pieces this was not a problem and I was aware that it became a problem for 7 pieces (I had actually mentioned it early in this thread), but then I forgot about it when I had modified the compressor to deal with >= 240 initial values when building the dictionary of phrases and things seemed to work.

So many thanks to Niklas for reporting this problem early!

[noobpwnftw]

Thank you. Now I understand more about the compression logic.

That table is at reconstruction step now, should finish soon.
Pawnless 5v2 is also about to finish(8 remaining), things look promising.

[noobpwnftw]

Regenerated KRBNvKQN has been uploaded.

[niklasf]

I tried to implement this in python-chess: https://github.com/niklasf/python-chess ... 5485510acd

It's deployed, but results still do not seem to be consistent: https://syzygy-tables.info/?fen=QN4n1/4 ... _w_-_-_0_1

I hope the tables are up to date:

Code: Select all

$ md5sum KRBNvKQN.rtb*
7b3be9c14ba4c9503e51e9f881552f55  KRBNvKQN.rtbw
5adb3f9e4f25bc47ae0194dc565ba164  KRBNvKQN.rtbz

I could easily imagine that I made a mistake with the pointer math, or some other oversight. I tried to compare to Cfish by merging master into the 7men branch, but

Code: Select all

setoption name syzygypath value /home/syzygy/tables/standard/3-4-5:/home/syzygy/tables/standard/6-wdl:/home/syzygy/tables/standard/7
position fen QN4n1/6r1/3k4/8/b2K4/8/8/8 b - - 0 1
d

yields

Code: Select all

make[1]: Leaving directory '/home/syzygy/syzygy-tables.info/Cfish/src'
Cfish 270418 64 NUMA by Syzygy based on Stockfish
info string Found 638 tablebases.

 +---+---+---+---+---+---+---+---+
 | Q | N |   |   |   |   | n |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   | r |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   | k |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 | b |   |   | K |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+

Fen: QN4n1/6r1/3k4/8/b2K4/8/8/8 b - - 0 1
Key: 8A40FB6371DE952F
Checkers:
Tablebases WDL:   -2 (1)
Tablebases DTZ: -450 (1)
Tablebases DTM: mate 0 (0)

with WDL -2, so I probably can't do it like that. Will continue to debug tomorrow, but I thought I'd share this as soon as possible.

[syzygy]

Thanks, I'm now downloading the two files myself to see if I can make sense of it.

[syzygy]

I have downloaded the two files. Their md5sums are identical to yours. tbcheck confirms that their internal checksums are correct.

The results returned by probing them are completely wrong.
Even the WDL results seem to be wrong:

Longest black win (100 ply) returns a draw:
Fen: 8/8/8/q7/8/8/1R3B2/NKnk4 w - - 0 1
Key: 3FF7F318447D441F
Checkers:
Tablebases WDL: 0 (1)
Tablebases DTZ: 0 (1)
Tablebases DTM: mate 0 (0)

Longest white win (65 ply) returns a win, but in just 10 ply:
Fen: 5q1n/8/8/4N3/4B3/3K2R1/8/4k3 w - - 0 1
Key: 8AF65EE2CD3390E8
Checkers:
Tablebases WDL: 2 (1)
Tablebases DTZ: 10 (1)
Tablebases DTM: cp 0 (0)

Longest white cursed win (152 ply) returns a white loss:
Fen: 5n2/8/8/7q/3N1R2/3B4/8/k2K4 w - - 0 1
Key: 99AC33FADD29C35A
Checkers: h5
Tablebases WDL: -2 (1)
Tablebases DTZ: -450 (2)
Tablebases DTM: mate 0 (0)

Longest black cursed win (1034 ply) returns a white loss/black win, but in just 454 ply:
Fen: 8/2B5/8/1k6/8/3K4/4N1R1/qn6 w - - 0 1
Key: 87683D8F8EA45EB4
Checkers:
Tablebases WDL: -2 (1)
Tablebases DTZ: -454 (1)
Tablebases DTM: mate 0 (0)

[syzygy]

OK, the WDL problems are caused by an overflow in the probing code:
https://github.com/syzygy1/Cfish/commit ... 5c67b9bc19

This is also a problem in the corresponding code of the generator, which means we have to make sure that none of the TBs generated so far can have been affected by this.

The pawnless tables should not be affected, since they depend only on 6-men tables, and they are smaller than 4GB so the overflow cannot occur.

The pawnful tables may be affected, but only if they depend on a 7-men table for which one of the wtm and btm subtables is bigger than 4GB.

It seems there are no such tables:
- all pawnful tables so far are 6v1 tables.
- all pawnless 6v1 tables are smaller than 4GB (KQQRBNvK.rtbw being the largest at just 1092431 KB).
- the only pawnful 6v1 tables larger than 4GB is KQRBNPvK.rtbw at 5087196 KB. It seems extremely unlikely than one of its 8 subtables is bigger than 4GB. But I will check this just to be sure.