Can Leela CPU train from a PGN file instead of selfplay?

[Jimbo I]

I'm hoping that it's possible for Leela CPU to train from a large PGN file instead of selfplay. If so, does anyone know how to explain it simply enough for an average Joe to understand it?

[Albert Silver]

The answer is a yes and no.

First, let me clarify what I think is probably a misunderstanding on your part, an understandable one, you cannot train the network directly from the binary+NN combination. You need to set up the training infrastructure on your computer in order to train a network, which involves CUDA, Python, Tensorflow, the Leela training codebase, and a few other dependencies. While definitely doable for the Average Joe (raises hand), there is a significant learning curve.

That said, you can train from a pre-existing Leela net, though it still requires the above. There is simply another step to create a training model/starting point from the net in question.

[Jimbo I]

Thanks for the answer. It sounds like it would be a little above my pay grade. Oh well, it was a nice thought.

[phhnguyen]

It is possible (we can do almost all things in software word if we really want) but that is not good enough:
- A typical PGN file set contains from so-so to good chess positions. However, for training purposes, Lc0 must be trained by both good and bad or even crazy positions - it learns from bad positions too. Clearly that PGN set can't cover enough chess positions. Without covering enough, Lc0 may even don't know how to move (or create illegal moves) in some situations. For a good covering, a random game generator is much better than using a PGN set
- The number of games in a typical PGN file set is so small (maybe a few million games) compared with a normal training set - at least a few tens million games for a small net. You will run out of games for training very soon and stop improving the strength of the program
- Basically a learner will use what he was trained to solve the problem and he always runs after but can't be better than his teachers (we are talking about learning programs, not creative learning human ). If you train Lc0 by games from a chess engine, say Stockfish, don't hope it can beat Stockfish one day

[the_real_greco]

As has been pointed out, one could START with a fully trained net, then feed it a lot of PGNs at the end.

The caveat being, you'd still need a huge number of games to change the net significantly. Or you'd have to crank up the learning rate so high you risk destroying all the initial knowledge. Or use a tiny, tiny network.

So if you want to make a network trained from PGNs, start with a small network, high learning rate, and a huge number of games. And cross your fingers.

[dkappe]

It is possible (we can do almost all things in software word if we really want) but that is not good enough:
- A typical PGN file set contains from so-so to good chess positions. However, for training purposes, Lc0 must be trained by both good and bad or even crazy positions - it learns from bad positions too. Clearly that PGN set can't cover enough chess positions. Without covering enough, Lc0 may even don't know how to move (or create illegal moves) in some situations. For a good covering, a random game generator is much better than using a PGN set
- The number of games in a typical PGN file set is so small (maybe a few million games) compared with a normal training set - at least a few tens million games for a small net. You will run out of games for training very soon and stop improving the strength of the program
- Basically a learner will use what he was trained to solve the problem and he always runs after but can't be better than his teachers (we are talking about learning programs, not creative learning human ). If you train Lc0 by games from a chess engine, say Stockfish, don't hope it can beat Stockfish one day

A small number of games will create a net that doesn’t play well, but illegal moves aren’t in the picture.

The smallest number of games I’ve trained with was 500k, augmenting the games by playing out non-terminals with sf11 to get enough endgame positions. I also augmented the game data with sf11 multipv data. The resulting 32x4 net was maybe 1 cpu 2900 ccrl. A bigger, 128x10 net trained in a similar manner on 2.5m games can reach 3100+ 1 cpu ccrl. Getting up to 3400+ requires a bit more.

[Ovyron]

I wonder what would happen if the entire Lichess database was used to train a neural network. It wouldn't play very strongly but could possibly play in a way indistinguishable from humans.

[Jimbo I]

I wonder what would happen if the entire Lichess database was used to train a neural network. It wouldn't play very strongly but could possibly play in a way indistinguishable from humans.

Actually, the reason for my question, except I was thinking about using only a thin ratings slice of the database. Would it then play roughly at that level?

Even if possible, someone else would have to do it - It's beyond my capabilities.

[phhnguyen]

A small number of games will create a net that doesn’t play well, but illegal moves aren’t in the picture.

Yes, theory it could play illegal moves, at least for older versions. Zero means the program has no knowledge about chess, including move rules. All will be learned from games. Suppose a given piece can move from cell A to B. If you train the program only by a small set of games, it may never see that move (A to B). When playing, suppose it reaches a position in which the move A->B is only a legal move, the program may not make that move (because it has never been taught). Instead, it may make randomly another move (based on its probability) or claim wrongly a draw (a kind of illegal move too).

As I have read for recent versions Lc0 has installed some chess knowledge, including move rules (thus it is not completely Zero) to speed up learning and it can avoid the above issue.

[Ovyron]

I was thinking about using only a thin ratings slice of the database.

Magic would happen (if it would happen) after you have trained her with as much games as you can, so by limiting them to a small slice you'd only hurt the training.

You need to find the biggest cake and feed it whole, that's why current methods just produce games to train her, not only to reach strength (which you don't care about) but because reaching anything useful requires HUGE numbers.