7-men Syzygy attempt

[Arpad Rusz]

Having the full set of 7-man Syzygy tablebases would be cool. In practice, some tablebases are very important for endgame analysis and some are likely just wasting storage place. I have created a heat map for the 4v3 tablebases. The bases with material differences of 1 or 3 are the most important and one should probably get all of them. I feel it is safe to ignore the tablebases with material differences bigger than 5.

[Sesse]

Is this heat map based on how many times these positions actually come up in games, or on the material difference alone?

[Sesse]

Also, this surely makes no sense. KPPPvKPP has material difference 1 in your table, and KPPPvKQP has material difference 8. According to your logic of ignoring everything with material differences larger than 5, you should keep KPPPvKPP, and then once the first pawn promotes, you're out of tablebase and have no idea how to play the rest of the game?

[Arpad Rusz]

Also, this surely makes no sense. KPPPvKPP has material difference 1 in your table, and KPPPvKQP has material difference 8. According to your logic of ignoring everything with material differences larger than 5, you should keep KPPPvKPP, and then once the first pawn promotes, you're out of tablebase and have no idea how to play the rest of the game?

I presume the engine alone will easily find the win if the material difference is so big. Although some tricky positions may exist.

[Sesse]

I presume the engine alone will easily find the win if the material difference is so big. Although some tricky positions may exist.

I don't think that assumption holds.

[syzygy]

Also, this surely makes no sense. KPPPvKPP has material difference 1 in your table, and KPPPvKQP has material difference 8. According to your logic of ignoring everything with material differences larger than 5, you should keep KPPPvKPP, and then once the first pawn promotes, you're out of tablebase and have no idea how to play the rest of the game?

I presume the engine alone will easily find the win if the material difference is so big. Although some tricky positions may exist.

But it is not possible to generate KPPPvKPP without first generating KPPPvKQP and KQPPvKPP (and KXPPvKPP and KPPPvKXP for X=R,B,N) unless you don't care about the correctness and completeness of the results. If you don't care about the correctness and completeness, then why bother at all.

Likewise, it is not possible to generate KQPPvKPP without first generating KQQPvKPP, and so on.

The end of the story is that to get KPPPvKPP, you need all other 4v3 tables first. So that's what being worked on.

As for the idea of downloading only the "useful" tables, if disk space is limited then obviously some choices will have to be made, but you will likely run into problems now and then. Also, the "non-useful" ones are typically much smaller than the "useful" ones (at least the WDL tables). Anyway, what you cannot do without is the full 6-piece 4v2 and 3v3 WDL if you plan on using some of the 4v3 tables.

[Nordlandia]

Does it make sense to store "non-useful" tables at HDD as DTZ. Important tables to be priorities on SSD.

[syzygy]

Does it make sense to store "non-useful" tables at HDD as DTZ. Important tables to be priorities on SSD.

No, unless you know that they will almost never be accessed. But in practice even the most unlikely tables seem to get accessed in certain endgames.

What would make sense is to limit the engine's probing of those tables that its search almost cannot get wrong. Houdini does something like that. Or one could let the engine figure out which tables are on SSD and which on HDD and adjust probing based on that. There is no end to optimisation.

[Nordlandia]

Does it make sense to store "non-useful" tables at HDD as DTZ. Important tables to be priorities on SSD.

No, unless you know that they will almost never be accessed. But in practice even the most unlikely tables seem to get accessed in certain endgames.

What would make sense is to limit the engine's probing of those tables that its search almost cannot get wrong. Houdini does something like that. Or one could let the engine figure out which tables are on SSD and which on HDD and adjust probing based on that. There is no end to optimisation.

Houdini's intelligent probing algorithm is comparable to how an floodgate operate. Water is tablebases, gate is egtb probe depth (intelligent probing). At 0 the gate is completely open, while at 5 the gate is half closed which means the tablebase flow is restricted by 50%.

[Sesse]

Houdini's intelligent probing algorithm is comparable to how an floodgate operate. Water is tablebases, gate is egtb probe depth (intelligent probing). At 0 the gate is completely open, while at 5 the gate is half closed which means the tablebase flow is restricted by 50%.

I am unable to make sense of that analogy in any reasonable way.