Tablebase suggestion

[Ajedrecista]

Hello:

I want to add something inside all this technical discussion:

Op1 - Partial 8-piece tablebase available

This is an article on Lichess blog posted today about the generation, storage and uploading of op1 partial 8-man EGTB.

Regards from Spain.

Ajedrecista.

[Jouni]

What is 7/8 piece benefit? Remainder:

TB6 on SSD

Results of Stockfish vs StockfishSyzygy (10+0.1, 1t, 16MB, UHO_XXL_+0.90_+1.19.pgn):
Elo: -1.26 +/- 1.46, nElo: -2.64 +/- 3.05
LOS: 4.46 %, DrawRatio: 57.40 %, PairsRatio: 0.96
Games: 50000, Wins: 13671, Losses: 13853, Draws: 22476, Points: 24909.0 (49.82 %)
Ptnml(0-2): [130, 5293, 14349, 5085, 143], WL/DD Ratio: 1.37

[syzygy]

Hello:

I want to add something inside all this technical discussion:

Op1 - Partial 8-piece tablebase available

This is an article on Lichess blog posted today about the generation, storage and uploading of op1 partial 8-man EGTB.

Interesting. So 9-piece op2 is indeed being worked on.

[Geezer one]

What is 7/8 piece benefit? Remainder:

TB6 on SSD

Results of Stockfish vs StockfishSyzygy (10+0.1, 1t, 16MB, UHO_XXL_+0.90_+1.19.pgn):
Elo: -1.26 +/- 1.46, nElo: -2.64 +/- 3.05
LOS: 4.46 %, DrawRatio: 57.40 %, PairsRatio: 0.96
Games: 50000, Wins: 13671, Losses: 13853, Draws: 22476, Points: 24909.0 (49.82 %)
Ptnml(0-2): [130, 5293, 14349, 5085, 143], WL/DD Ratio: 1.37

Engine strength is highly dependant on Neural Nets. They were trained with tablebase data included. 8 piece tablebase could improve that training data.

[syzygy]

The first half of the iteration is:

Code: Select all

S1 = Load(W, WIN_IN_N(n));
R1 = Predecessor(S1);
S2 = Load(B, UNKNOWN);
S2 = S2 & R1;
----
R2 = Load(W, WIN_IN<=N(n));
S3 = ProveSuccessor(S2, R2);
B = Add(S3, LOSE_IN_N(n));
----
R3 = Predecessor(S3);
S4 = Load(W, UNKNOWN);
S4 = S4 & R3;
W = Add(S4, WIN_IN_N(n+1));

(and then we do the same steps with W and B reversed)

Instead of working with intermediate "W" and "B" tables to and from which WIN/LOSE_IN_N(n) positions are added and extracted during each iteration, it seems a lot more efficient to keep everything in bitmaps and only merge them at the end. This probably requires more disk space for intermediate files, but if disk space is limited then generating 8-piece tables is a very silly idea to begin with.

So the goal is to generate the WIN_IN_N(n) and LOSE_IN_N(n) bitmaps for both wtm and btm.
To do this, the algorithm also needs to maintain UNKNOWN and WIN_IN<=N(n) bitmaps (for both wtm and btm).

However, what is the UNKNOWN bitmap used for exactly:

Code: Select all

S1 = Load(W, WIN_IN_N(n));
R1 = Predecessor(S1);
S2 = Load(B, UNKNOWN);
S2 = S2 & R1;
----
R2 = Load(W, WIN_IN<=N(n));
S3 = ProveSuccessor(S2, R2);
B = Add(S3, LOSE_IN_N(n));

R1 is the set of potential btm LOSE_IN_N(n) positions, which are the positions from which a btm move leads to a wtm WIN_IN_N(n) position.
So we already know that R1 will not contain any LOSE_IN_N(k) positions for k < N.
So the AND operation with UNKNOWN can only remove illegal positions and positions which have been determined to be a win.
The set of positions that have been determined to be a win corresponds to the WIN_IN<=N(n) bitmap.
What if we include in the WIN_IN_<=N(n) bitmap the set of ILLEGAL positions. Then we don't need the UNKNOWN bitmap to generate LOSE_IN_N(n).
(So instead of AND with UNKNOWN, we AND with NOT WIN_IN_<=N(n).)

Now what about these two lines:

Code: Select all

R2 = Load(W, WIN_IN<=N(n));
S3 = ProveSuccessor(S2, R2);

If we include the ILLEGAL positions in the WIN_IN<=N bitmap, then ProveSuccessor(S2,R2) still works correctly. A move from a potential loss in S2 into an illegal position is an illegal move and therefore does not refute the loss. In fact, we can now dispense with checking move legality, which is great. We actually want the ILLEGAL positions to be included in WIN_IN<=N(n) here anyway!

The other part is:

Code: Select all

R3 = Predecessor(S3);
S4 = Load(W, UNKNOWN);
S4 = S4 & R3;
W = Add(S4, WIN_IN_N(n+1));

R3 is the set of position wtm positions for which a move leads to a btm LOSE_IN_N(n). Since these positions have a winning move, they cannot be losing positions.
So, again, the AND operation with UNKNOWN can only remove illegal positions and positions which have been determined earlier to be a win.
So we can again replace UNKNOWN with the bitmap that is the union of ILLEGAL and WIN_IN<=N(n).

So maintaining an UNKNOWN bitmap does not seem necessary. We only need to maintain wtm and btm bitmaps of illegal plus winning positions. This is a very welcome simplification and efficiency gain.