Using Freeware AI and Dynamically Generated Endgame Tablebases

[zug234zwang]

The purpose of this query is to discuss a topic that I am totally ignorant of, rather than pose one specific question. Although such a query does invite opinions based on intuition, I am also looking for objective facts based on people's experience.

I am wondering whether anyone has begun combining freeware AI with dynamically generated endgame tablebases to attack chess studies.

The following link discusses freeware chess AI:
https://en.wikipedia.org/wiki/Leela_Chess_Zero

The following link discusses chess AI in general:
https://chess.stackexchange.com/questio ... ss-ai?rq=1

The following link may relate to building a tablebase on-the-fly to attack a specific position:
http://kirill-kryukov.com/chess/discuss ... 108#p16108

The general algorithm for constructing a tablebase is discussed here:
http://www.talkchess.com/forum3/viewtopic.php?t=51847

Suppose (for example) Leela Chess Zero (with standard 6 man tablebases) was used to play N games against itself, always starting from a specific position. Assume that Leela Chess Zero (somehow) learns from each game and stores the knowledge. The (hopefully plausible) intent is that by playing these games, Leela Chess Zero acquires pertinent insights that allow it to determine "best play".

The following links suggest sample positions to attack:
https://chess.stackexchange.com/questio ... -four?rq=1
https://chess.stackexchange.com/questio ... l-to-crack
https://chess.stackexchange.com/questio ... ect=1&lq=1

Can such an approach be implemented on a modern, moderately priced home pc?
To attack a typical position:
1. How large should N be (i.e. how many games played) to give Leela Chess Zero a plausible chance of "learning" insights into the position?
2. What time controls should be employed and how long will the process take?
3. How much memory (? + GPU memory) should be employed?
4. How strong a cpu (? + GPU-processor) should be employed?

Assuming that the above issues are resolved, how does one determine what the insights are? Would it be possible to identify a position that Leela Chess Zero categorizes as both obtainable and "critical"?

Assume that Leela identifies a specific (critical and obtainable) position via its FEN. Would it be possible to build a tablebase (from scratch) with the critical position as a starting point, so that only those positions that might legally result from this starting point are relevant?

Suppose that the approach described in the previous paragraph was feasible, with a clear determination reached of [win for white, win for black, or draw]. Could these results be fed back into Leela Chess Zero so that it used this result + all the learning that it had achieved on the position, to start a new series of N games (with Leela Chess Zero playing itself) from the original starting position?

Is it plausible that the iterative process of combining Leela Chess Zero with dynamically generated tablebases could reach a definitive conclusion of (for example) a complex middle game with many pieces?

[Dann Corbit]

LC0 already uses tablebase files.
Generation on the fly isn't necessary.

[hgm]

Well, it would be for 9-men end-games. It doesn't use those yet...

[zug234zwang]

Especially since a home user might not want to expend the hard disk space for more than 6man tb's. I imagine a home user setting up leela chess zero with 6 man tablebases and then (somehow) beginning the iterative process.

[Dann Corbit]

There is no savings of any kind for building 6 man tablebase files on the fly.
And I really mean it.

[zug234zwang]

I didn't explain myself well. My intent is the user has Leela Chess Zero (for example) begin with the standard 6 man tablebases, and based on that (hopefully, somehow) Leela identifies (for example) 2 or 3 critical positions that each have (for example) 12 men.

Then you manually examine each position and (hopefully) identify (perhaps) 10 different positions, each with exactly 10 men. Then for each position, you build (on-the-fly) a 10 man tablebase that only considers positions that can legally arise from the specific position.

This means that you will end up with 10 separate 10 man tablebases, each of whom is readable to Leela. You connect these tablebases to Leela (hopefully as well as the standard 6 man tablebases), and then have Leela begin the second round of iteration.

Perhaps I am mistaken. Perhaps you already understood that this was my intent and are saying that "it won't be worth it." If so, please advise. If not, is this approach do-able? If so, is anyone doing it?

[Dann Corbit]

I think the approach you are considering is Eiko Bleicher's "Freezer"
At one time it was an open source project.
Now you can buy it from here:
http://www.freezerchess.com/index.php?topic=download
There is an open source program that does the same thing called Hoffman. You can find it on github.

[noobpwnftw]

There is a conceptional difference between "an exhaustive enumeration of subsequent positions given a certain position" and "an exhaustive enumeration of every possible positions given a certain set of material". The former is essentially a search without any pruning, functionally being a special case of the latter.

I think FinalGen is also one of which does that. Note that unless the extra pieces you add are really meaningless, they would still result in a significant amount of positions to look into.

[Dann Corbit]

There is a conceptional difference between "an exhaustive enumeration of subsequent positions given a certain position" and "an exhaustive enumeration of every possible positions given a certain set of material". The former is essentially a search without any pruning, functionally being a special case of the latter.

I think FinalGen is also one of which does that. Note that unless the extra pieces you add are really meaningless, they would still result in a significant amount of positions to look into.

Finalgen tends to be useful when opposing pawns are.locked.

[hgm]

It should even be useful without locked Pawns. The usefulness comes from limiting the EGT to the reachable Pawn constellations, rather than all possible constellations with a given number of Pawns. Even a white passer on e2 can only reach 1/8 of all positions that could exist with a single Pawn. If the passer is already advanced, that number goes down further.

It is true that locked Pawns tend to give a larger reduction than unlocked Pawns, especially if they are already head to head. Then it can be almost like they are not there at all. Fortunately locked Pawns are a very common occurrence in games.