I updated the code. Refactored a lot of it to try to make it easier to use in multiple different games and have some more standard search features and faster neural net calculation.
GuiNN 2.04 against GuiCheckers 1.11 (an old pre-NN version) at .2s per-move
1169-31-1680 W-L-D (+145 elo)
Which is pretty dominating for checkers. It still lost some games, but I expect if it had better training (and more effort making sure search is strong and bug-free) it would be very hard to beat.
I did a first pass on incremental update of first layer, which I think was a small gain in 8x8 checkers (like 5% faster in early game trailing off to even speed.) I think incremental will be more useful in some other games, and also would have bigger impact if I was evaluating every board during search like is pretty common instead of just the final boards.
However it did get me to start to wonder why the rest of the net was as slow as it was. It turns out that even simple stuff like relu activation clamping, and to lesser extent the converting the 32-bit intermediate values to 16-bit fixed values, can cause a big slowdown when using regular code and poorly mixed in to the program flow. I converted all this stuff to SIMD instructions done on a full array for a 20% speed gain. One thing pretty nice about general stuff like NNs is improving multiple programs at once. After pasting the neuralNet cpp files into Slow Chess the endgame test versus SF11 was -27 now.
First success with neural nets
Moderators: hgm, Rebel, chrisw
-
jonkr
- Posts: 178
- Joined: Wed Nov 13, 2019 1:36 am
- Full name: Jonathan Kreuzer
-
Rein Halbersma
- Posts: 749
- Joined: Tue May 22, 2007 11:13 am
Re: First success with neural nets
Over on a computer draughts forum, I've posted about a Keras model that I wrote that managed to improve the 10x10 draughts program Kingsrow by 1.2 Elo during its very first training run. See http://laatste.info/bb3/viewtopic.php?f=53&t=8327
Note that in 10x10 draughts, the currently dominant eval structures are based on Fabien Letouzey's pattern invention that he introduced in 2015 with his open source draughts program Scan. Ed Gilbert's program Kingsrow is one of those top programs that uses such patterns. Ed provided me with 231 million training positions, that was a great help!
There is at least one guy (Bert Tuyt) experimenting with porting your NN from 8x8 checkers to 10x10 draughts. So it's an exciting time for games programming in general
I used the Keras Functional API rather than the Keras Sequential API because of the 3 separate input layers for that type of evaluation function. In any case, your post on applying Keras to checkers is what inspired me to try and master this technology myself, so thanks! I'm open sourcing this model in the coming days after it's been polished a bit more.
Note that in 10x10 draughts, the currently dominant eval structures are based on Fabien Letouzey's pattern invention that he introduced in 2015 with his open source draughts program Scan. Ed Gilbert's program Kingsrow is one of those top programs that uses such patterns. Ed provided me with 231 million training positions, that was a great help!
There is at least one guy (Bert Tuyt) experimenting with porting your NN from 8x8 checkers to 10x10 draughts. So it's an exciting time for games programming in general
I used the Keras Functional API rather than the Keras Sequential API because of the 3 separate input layers for that type of evaluation function. In any case, your post on applying Keras to checkers is what inspired me to try and master this technology myself, so thanks! I'm open sourcing this model in the coming days after it's been polished a bit more.
-
jonkr
- Posts: 178
- Joined: Wed Nov 13, 2019 1:36 am
- Full name: Jonathan Kreuzer
Re: First success with neural nets
Rein :
Yeah Tensorflow is pretty cool to work/experiment with, I just started trying out more stuff (was running chess training games for a while, but started to feel tedious again so went back to trying out various experiments with learning.) I finally installed cuda drivers for my gpu and now training is 4 times faster (I spend so much more time on generating games I hadn't bothered, but it makes it more fun to try adjusting/experimenting with the tensorflow side with it faster.)
Went back to work on generalizing the system, and am trying to test the generalizing in Othello. It's a good game to try out fully automated learning from self-play, given possibility of searching for perfect endgames then with more training the early stages get better too. It seems less suited for a board-input neural net, but that might be good for experimenting and learning something. It's already good but haven't tested against my old pattern program to see if it has chance of being better.
I haven't decided yet if I will try out 10x10 draughts, I am curious how it would go, and if anyone else tries it what results they get. My instinct is that board inputs would perform quite well for that too as the complexity would be above checkers, but still quite a bit below chess (for draughts 50 usable squares, 2 piece types per color.) Training would definitely take longer but wonder how much longer.
For chess endgame nets the latest score was -9 elo to SF11 in the endgames.epd, for a 20,000 game match. The individual nets (with exception of QvR(s)+p which hopefully just needs more training) all appeared to perform equal or better than SF11 even with Slow's weaker search, but the general net for 2 or less piece per-side endgames is having trouble matching that perf. I'm not sure if it just needs a ton of data, or the method is reaching limitations as the game-space expands, or maybe search matters bit more. I changed the first layer to 256 for all nets, now that computation is faster and I have more data, that seemed a few elo better. I have 18 million endgame training positions overall. I've taken another break from chess endgames as the progress is feeling slow again, but out of pure stubborness I expect I will go back eventually to try to get at least some small win in a 10k+ game match.
Yeah Tensorflow is pretty cool to work/experiment with, I just started trying out more stuff (was running chess training games for a while, but started to feel tedious again so went back to trying out various experiments with learning.) I finally installed cuda drivers for my gpu and now training is 4 times faster (I spend so much more time on generating games I hadn't bothered, but it makes it more fun to try adjusting/experimenting with the tensorflow side with it faster.)
Went back to work on generalizing the system, and am trying to test the generalizing in Othello. It's a good game to try out fully automated learning from self-play, given possibility of searching for perfect endgames then with more training the early stages get better too. It seems less suited for a board-input neural net, but that might be good for experimenting and learning something. It's already good but haven't tested against my old pattern program to see if it has chance of being better.
I haven't decided yet if I will try out 10x10 draughts, I am curious how it would go, and if anyone else tries it what results they get. My instinct is that board inputs would perform quite well for that too as the complexity would be above checkers, but still quite a bit below chess (for draughts 50 usable squares, 2 piece types per color.) Training would definitely take longer but wonder how much longer.
For chess endgame nets the latest score was -9 elo to SF11 in the endgames.epd, for a 20,000 game match. The individual nets (with exception of QvR(s)+p which hopefully just needs more training) all appeared to perform equal or better than SF11 even with Slow's weaker search, but the general net for 2 or less piece per-side endgames is having trouble matching that perf. I'm not sure if it just needs a ton of data, or the method is reaching limitations as the game-space expands, or maybe search matters bit more. I changed the first layer to 256 for all nets, now that computation is faster and I have more data, that seemed a few elo better. I have 18 million endgame training positions overall. I've taken another break from chess endgames as the progress is feeling slow again, but out of pure stubborness I expect I will go back eventually to try to get at least some small win in a 10k+ game match.