Can Leela CPU train from a PGN file instead of selfplay?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Jimbo I
Posts: 148
Joined: Thu Feb 15, 2007 3:34 am
Location: USA

Can Leela CPU train from a PGN file instead of selfplay?

Post by Jimbo I » Wed Mar 04, 2020 5:06 pm

I'm hoping that it's possible for Leela CPU to train from a large PGN file instead of selfplay. If so, does anyone know how to explain it simply enough for an average Joe to understand it?

Albert Silver
Posts: 2894
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by Albert Silver » Wed Mar 04, 2020 6:17 pm

The answer is a yes and no.

First, let me clarify what I think is probably a misunderstanding on your part, an understandable one, you cannot train the network directly from the binary+NN combination. You need to set up the training infrastructure on your computer in order to train a network, which involves CUDA, Python, Tensorflow, the Leela training codebase, and a few other dependencies. While definitely doable for the Average Joe (raises hand), there is a significant learning curve.

That said, you can train from a pre-existing Leela net, though it still requires the above. There is simply another step to create a training model/starting point from the net in question.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Jimbo I
Posts: 148
Joined: Thu Feb 15, 2007 3:34 am
Location: USA

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by Jimbo I » Wed Mar 04, 2020 9:53 pm

Thanks for the answer. It sounds like it would be a little above my pay grade. Oh well, it was a nice thought. :)

User avatar
phhnguyen
Posts: 719
Joined: Wed Apr 21, 2010 2:58 am
Location: Australia
Full name: Nguyen Hong Pham
Contact:

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by phhnguyen » Thu Mar 05, 2020 1:40 am

It is possible (we can do almost all things in software word if we really want) but that is not good enough:
- A typical PGN file set contains from so-so to good chess positions. However, for training purposes, Lc0 must be trained by both good and bad or even crazy positions - it learns from bad positions too. Clearly that PGN set can't cover enough chess positions. Without covering enough, Lc0 may even don't know how to move (or create illegal moves) in some situations. For a good covering, a random game generator is much better than using a PGN set
- The number of games in a typical PGN file set is so small (maybe a few million games) compared with a normal training set - at least a few tens million games for a small net. You will run out of games for training very soon and stop improving the strength of the program
- Basically a learner will use what he was trained to solve the problem and he always runs after but can't be better than his teachers (we are talking about learning programs, not creative learning human ;) ). If you train Lc0 by games from a chess engine, say Stockfish, don't hope it can beat Stockfish one day
https://banksiagui.com
A freeware chess GUI, based on opensource Banksia - the chess tournament manager

the_real_greco
Posts: 33
Joined: Thu Sep 19, 2019 3:55 am
Full name: Andy Erly

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by the_real_greco » Thu Mar 05, 2020 3:23 am

As has been pointed out, one could START with a fully trained net, then feed it a lot of PGNs at the end.

The caveat being, you'd still need a huge number of games to change the net significantly. Or you'd have to crank up the learning rate so high you risk destroying all the initial knowledge. Or use a tiny, tiny network.

So if you want to make a network trained from PGNs, start with a small network, high learning rate, and a huge number of games. And cross your fingers.

dkappe
Posts: 371
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by dkappe » Thu Mar 05, 2020 3:49 am

phhnguyen wrote:
Thu Mar 05, 2020 1:40 am
It is possible (we can do almost all things in software word if we really want) but that is not good enough:
- A typical PGN file set contains from so-so to good chess positions. However, for training purposes, Lc0 must be trained by both good and bad or even crazy positions - it learns from bad positions too. Clearly that PGN set can't cover enough chess positions. Without covering enough, Lc0 may even don't know how to move (or create illegal moves) in some situations. For a good covering, a random game generator is much better than using a PGN set
- The number of games in a typical PGN file set is so small (maybe a few million games) compared with a normal training set - at least a few tens million games for a small net. You will run out of games for training very soon and stop improving the strength of the program
- Basically a learner will use what he was trained to solve the problem and he always runs after but can't be better than his teachers (we are talking about learning programs, not creative learning human ;) ). If you train Lc0 by games from a chess engine, say Stockfish, don't hope it can beat Stockfish one day
A small number of games will create a net that doesn’t play well, but illegal moves aren’t in the picture.

The smallest number of games I’ve trained with was 500k, augmenting the games by playing out non-terminals with sf11 to get enough endgame positions. I also augmented the game data with sf11 multipv data. The resulting 32x4 net was maybe 1 cpu 2900 ccrl. A bigger, 128x10 net trained in a similar manner on 2.5m games can reach 3100+ 1 cpu ccrl. Getting up to 3400+ requires a bit more.

User avatar
Ovyron
Posts: 4002
Joined: Tue Jul 03, 2007 2:30 am

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by Ovyron » Thu Mar 05, 2020 4:08 am

I wonder what would happen if the entire Lichess database was used to train a neural network. It wouldn't play very strongly but could possibly play in a way indistinguishable from humans.

Jimbo I
Posts: 148
Joined: Thu Feb 15, 2007 3:34 am
Location: USA

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by Jimbo I » Thu Mar 05, 2020 4:36 am

Ovyron wrote:
Thu Mar 05, 2020 4:08 am
I wonder what would happen if the entire Lichess database was used to train a neural network. It wouldn't play very strongly but could possibly play in a way indistinguishable from humans.
Actually, the reason for my question, except I was thinking about using only a thin ratings slice of the database. Would it then play roughly at that level?

Even if possible, someone else would have to do it - It's beyond my capabilities. :(

User avatar
phhnguyen
Posts: 719
Joined: Wed Apr 21, 2010 2:58 am
Location: Australia
Full name: Nguyen Hong Pham
Contact:

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by phhnguyen » Thu Mar 05, 2020 5:10 am

dkappe wrote:
Thu Mar 05, 2020 3:49 am
A small number of games will create a net that doesn’t play well, but illegal moves aren’t in the picture.
Yes, theory it could play illegal moves, at least for older versions. Zero means the program has no knowledge about chess, including move rules. All will be learned from games. Suppose a given piece can move from cell A to B. If you train the program only by a small set of games, it may never see that move (A to B). When playing, suppose it reaches a position in which the move A->B is only a legal move, the program may not make that move (because it has never been taught). Instead, it may make randomly another move (based on its probability) or claim wrongly a draw (a kind of illegal move too).

As I have read for recent versions Lc0 has installed some chess knowledge, including move rules (thus it is not completely Zero) to speed up learning and it can avoid the above issue.
https://banksiagui.com
A freeware chess GUI, based on opensource Banksia - the chess tournament manager

User avatar
Ovyron
Posts: 4002
Joined: Tue Jul 03, 2007 2:30 am

Re: Can Leela CPU train from a PGN file instead of selfplay?

Post by Ovyron » Thu Mar 05, 2020 6:30 am

Jimbo I wrote:
Thu Mar 05, 2020 4:36 am
I was thinking about using only a thin ratings slice of the database.
Magic would happen (if it would happen) after you have trained her with as much games as you can, so by limiting them to a small slice you'd only hurt the training.

You need to find the biggest cake and feed it whole, that's why current methods just produce games to train her, not only to reach strength (which you don't care about) but because reaching anything useful requires HUGE numbers.

Post Reply