Re: So, how many of you are working on neural networks for chess?
Posted: Sun Feb 03, 2019 4:20 am
Given a game state S and the result of the game R in { -1.0 (loss) , 0.0 (tie), 1.0 (win) }, is the value head trained to predict R?brianr wrote: ↑Sun Feb 03, 2019 1:45 amIn terms of the actual net training process, Leela learns pretty much the same way with PGNs. The major practical difference is that the PGN game positions must first be converted into the input bit planes which are fed to the Leela net training code (Python and TensorFlow). Fortunately, the sample dataset (11GB compressed) includes both the PGNs and the bit plane files (2MM, IIRC, one file for each game), so it is much easier. The early versions of the SL code to convert PGN games to the bit plane format were a bit quirky, although I think there are some forks that might have fixed it. With the sample dataset that part can be skipped.
Generally, when learning from self-play games it is reinforcement learning (RL), and from PGNs it is supervised learning (SL). Which PGN games are selected and the order in which they are fed to the training process can have a significant impact on the resulting net strength. SL can train a net faster, but there is some debate about the ultimate result being as strong as RL. The massive crowd-sourced Leela training effort is primarily to provide self-play games for RL, and some match testing. Unfortunately, early on that was also called "training", which is not the same thing as using Tensorflow to actually train the weights in the net. Quite a few folks are also experimenting with SL.