Neural Net Endgame technique

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Alexander Lim
Posts: 32
Joined: Sun Mar 10, 2019 12:16 am
Full name: Alexander Lim

Neural Net Endgame technique

Post by Alexander Lim » Sun Jun 02, 2019 4:58 am

So can a neural net play the endgame or not? There seems to be allot of discussion going on with strong opinions and different criteria on what it means for a neural net to play the endgame. For example do we need it to be able replicate an endgame table base or just be able to finish off an endgame in a reasonable manner? Can it be an 'endgame only' NN or does it have to be proficient at all stages of play? And what about the trolling....

One thing missing from the discussions are actual examples of endgames being 'solved' by a neural net so people just assume Leela's endgame weaknesses are a neural net feature. I hope to rectify this in this post and hopefully any future discussions might take these examples into consideration.

(I'm aware of the efforts of the Ender project and was impressed with its ability to solve the KQ vs kr though it was unclear whether it can do, for example, NB or BB as well. Perhaps the author could comment?)

Anyway here are some examples of some standard 3 and 4 piece endgames. The original ChessFighter couldn't solve KNB vs k or KQ vs kr but later nets are able to (though it's still a little shaky sometimes). The nets used are all 'proper' nets in the sense that they can play the opening and middle game as well. The fact that they can do the endgame as well is just by the way.

Stockfish is not using any table bases though it gives a fairly optimal defence. ChessFighter is thinking for around 4-6 seconds per move at approx 6000 nps.

This message will be split up into multiple posts...

(Click on the 3 dots ... to see all the examples)

KNB vs k

Alexander Lim
Posts: 32
Joined: Sun Mar 10, 2019 12:16 am
Full name: Alexander Lim

Re: Neural Net Endgame technique

Post by Alexander Lim » Sun Jun 02, 2019 5:13 am

KQ vs kr

(Click on the 3 dots ... to see all the examples)


Alexander Lim
Posts: 32
Joined: Sun Mar 10, 2019 12:16 am
Full name: Alexander Lim

Re: Neural Net Endgame technique

Post by Alexander Lim » Sun Jun 02, 2019 5:22 am

KBB vs k


jp
Posts: 701
Joined: Mon Apr 23, 2018 5:54 am

Re: Neural Net Endgame technique

Post by jp » Mon Jun 17, 2019 9:02 am

Alexander Lim wrote:
Sun Jun 02, 2019 4:58 am
One thing missing from the discussions are actual examples of endgames being 'solved' by a neural net so people just assume Leela's endgame weaknesses are a neural net feature. I hope to rectify this in this post and hopefully any future discussions might take these examples into consideration.

(I'm aware of the efforts of the Ender project and was impressed with its ability to solve the KQ vs kr though it was unclear whether it can do, for example, NB or BB as well. Perhaps the author could comment?)
Alexander, what are the differences between your engine & Leela that might be causing different endgame performance? e.g. Are you training it the same way? (For Ender, it's obvious what the training differences are.)

User avatar
hgm
Posts: 23468
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Neural Net Endgame technique

Post by hgm » Mon Jun 17, 2019 9:31 am

A neural net of 3 neurons should already be enough (combined with PUCT search) to easily win KBNK. All that it has to know is that you have to drive the bare King into the corner, and which corner. For KBBK even a single neuron should be enough.

brianr
Posts: 352
Joined: Thu Mar 09, 2006 2:01 pm

Re: Neural Net Endgame technique

Post by brianr » Mon Jun 17, 2019 11:16 am

I did some lone king training with small nets in September last year. It was with a version of Leela that might still have had the 50 move draw issue. In any case, an 8x1 net was fine for KQ or KR v lone K. However, for KBNvK an 8x1 net could not mate. Needed to be at least 16x2 for about 2/3rds wins. But only trained with about 10% of entire sample set of all possible KBNvK positions. I also tried restricting the lone king to just one quadrant to reduce the possible combinations from nearly 11 M to 2.7. Generating test positions, training, and evaluating took about a week for each endgames, so by KBN I was testing with less than 200K games and probably many more samples are needed.

While I agree a far smaller net should be capable of mating with KBNvK, in practice training one was vastly more difficult than for the KQ and KR cases. Also, there was quite a bit of work on "ender" endgame nets using tablebases to train with something like 16 or fewer pieces that were quite quite successful, but these were much larger nets, IIRC. Look for posts by user dkappe.

Alexander Lim
Posts: 32
Joined: Sun Mar 10, 2019 12:16 am
Full name: Alexander Lim

Re: Neural Net Endgame technique

Post by Alexander Lim » Sun Jun 30, 2019 6:59 pm

jp wrote:
Mon Jun 17, 2019 9:02 am
Alexander Lim wrote:
Sun Jun 02, 2019 4:58 am
One thing missing from the discussions are actual examples of endgames being 'solved' by a neural net so people just assume Leela's endgame weaknesses are a neural net feature. I hope to rectify this in this post and hopefully any future discussions might take these examples into consideration.

(I'm aware of the efforts of the Ender project and was impressed with its ability to solve the KQ vs kr though it was unclear whether it can do, for example, NB or BB as well. Perhaps the author could comment?)
Alexander, what are the differences between your engine & Leela that might be causing different endgame performance? e.g. Are you training it the same way? (For Ender, it's obvious what the training differences are.)
Sorry for the delay in replying and thank you for your interest.

In my opinion there are two main reasons for Leela's apparent poor endgame performance.

Firstly Leela's evaluations are saturating to the extremes (+1, -1) to early so a mate in 1 position will have a similar evaluation to say a mate in 15 position. This causes funny behaviour like giving up material and general trolling around as it is happy with its +200 score.

Secondly given a position with say mate in 3 for which Leela can 'clearly see' the mate, it may not go for it if there are other moves with say a mate in 4. This is why it will go for the mate when the fifty move rule approaches as the alternative (longer) mates suddenly become draws. So Leela mating just before the 50 rule has nothing to do with the 50 rule itself, rather the impending draw forces Leela to go for the mate.

Chess Fighter has solved both problems and this allows it to 'learn' endgames and I also think it speeds up the learning process as a whole. I can't go into how I solved these issues as it will give away some of my techniques. However I can roughly describe what I did.

The original Chess Fighter couldn't solve the KNBk or KQkr endgames. Turns out the reason was because it hadn't been sufficiently exposed to the relevant endgame positions. During self play generation I run 50 parallel games at the same time from which I then dedicated 2 'workers' for KBNk, 2 for KQkr and 1 for KBBk. The dedicated worker then generated, for example, a random KBBk position and played on from there until mate or a draw, then started over. In particular I didn't use any table bases, it was all learnt from self-play. Once it learns to solve a particular endgame you can then reduce the numbers of workers until you have just one worker for all the tough endgames. I think you need to maintain at least one dedicated worker for endgame generation otherwise the net will eventually forget things.

So my solution did not involve any learning from a table base. Rather it was just 'normal' self-play games for which I had to force certain positions as the normal randomisation process (be it temperature or other methods) was not sufficient. Whenever I discover it can't solve a particular endgame I can then set a dedicated worker to randomly generate the particular positions and do a normal self play from there.

For all other basic endgames (as far as I'm aware) I didn't need to generate extra positions as they appeared in sufficient quantity during normal game generation.

* regarding Leela's eval saturating to the extremes. According to the Leela forums its actually a desired property and self-play resignations causes it more. I presume this makes it more sensitive to the changes from say a drawn position to a winning position. But once it gets to the winning position it's not as sensitive to changes to get to the final winning positions.

* if the Leela people implemented the above, it still wouldn't work! In fact their use of resignation is correct for the way Leela is trained otherwise you end up with KBBk, KNBk positions evaluated close to zero as I have seen on the old Leela 6x64 nets.

* My main aim is not to do anything with endgames and my recent techniques seem to have drowned out the generated endgame data and it can't solve them as well (though the resulting nets are around 300 ELO stronger!). I think the way forward will be to use table bases once I figure out how to use them.

* It's not clear whether Chess Fighters overall strength was increased or decreased by training it to solve certain endgames. It might be argued that by not learning them it could learn the middle game better. On the other hand, knowing how to play the endgame well may have a subtle positive affect on its overall play.

Related to this discussion are positions of extreme material imbalance for which it has been said that MCTS has issues with (due to the saturation of the eval function). Chess Fighter doesn't have any issues here:


noobpwnftw
Posts: 333
Joined: Sun Nov 08, 2015 10:10 pm

Re: Neural Net Endgame technique

Post by noobpwnftw » Sun Jun 30, 2019 7:21 pm

There is nothing wrong with evaluating a mate in 1 and mate in 10 equally in terms of winning probabilities.
Cosmetic "fixes" (as long as averaging and heuristics involved) without the ability to precisely discover the mating sequence is just as lame as it currently is. In other words, if the eval function is replaced with an oracle providing the ground truth and the outcome of the search algorithm is still based on luck, then to what end should one improve the eval function?

Post Reply