a crying shame (re: self-learning engines)

[carldaman]

The more I think about it, the more I'm convinced it's quite a bit of a travesty that these new self-learning programs actually do not support live learning!

I've recently run a match between the freeware Critter 1.6a, with its session file learning switched on, and the much-vaunted Fat Fritz, a commercial program I had to pay for, and it took less that 10 games for Critter to turn the match score around in its favor, due to its active learning, and trounce its GPU-based 'stronger' opponent. Of course, no real surprise here, but I get the feeling most people don't get this basic fact straight, that learning can steadily and significantly increase playing (and analysis) strength.

Yet, I haven't seen much discussion, let alone requests, about letting Leela (or its derivatives) continuing to learn as it plays. Is it that big a deal to change the program so this is made possible? It is puzzling and deflating to see Fat Fritz not being able to continue to learn while a free program can.

OK, as some might say, why should they bother, since they (the Leela team) do this work for free? Well, in that case why won't Chessbase? They surely have the resources to add such a useful feature! In either case, I think it's a poor argument to say that it would be difficult, since a lot more work has gone into prior coding, and adding such a learning feature is a guaranteed instant winner, and worth adding, even with serious effort.

The same thing goes for Komodo, or any other commercial endeavor playing 'catch-up' with stronger engines. Learning automatically and quickly adds strength. It's a simple fact. Can anyone argue with a straight face that they don't care about engine strength? This is what Komodo's programmers imply when they claim that there is just "not enough interest" in this feature. Not enough interest in far greater engine strength?! Cue laugh track, please...

Anyway, this post is not really directed at Komodo's programmers, but rather at Leela and Fat Fritz, much hyped engines that are based on self-learning, yet they can't learn a lick after you've installed them. If you still aren't convinced that learning works exceedingly well, check out Richard Vida's Critter or even Michael Sherwin's Romichess, both freeware, and the latter even open source.

[Leo]

I never had that kind of success with Critter learning. I will try again. Yes, some of us are interested in an engine that learns. Have you tried Brainlearn? I am slowly starting to think that LCZ may have plateaued.

[Daniel Shawul]

Lc0 is pre-trained with 40-million games already so it doesn't need it. In fact, you could easily screw up what is learned
by trying to adapt to your opponents, while the chance of you actually improving the net with "live learning" of a few hundred games is almost zero.
It is trained until it plateaus, to a point where there is almost no further elo gain (2e-5 learning rate or so...)

[carldaman]

OK, then what's wrong with learning to improve on bad lines, or faulty tactics, due to which it still loses games. I mean, it does lose games occasionally, right? That could only help its strength, and not hurt it, unless I'm missing something.

[carldaman]

I never had that kind of success with Critter learning. I will try again. Yes, some of us are interested in an engine that learns. Have you tried Brainlearn? I am slowly starting to think that LCZ may have plateaued.

I've tried Brainlearn, but I'm not yet convinced of how well it works. In fact, there are those who suggest that its learning is better suited for analysis and less for playing. I'm not yet certain of that, either way.

[Robert Flesher]

The more I think about it, the more I'm convinced it's quite a bit of a travesty that these new self-learning programs actually do not support live learning!

I've recently run a match between the freeware Critter 1.6a, with its session file learning switched on, and the much-vaunted Fat Fritz, a commercial program I had to pay for, and it took less that 10 games for Critter to turn the match score around in its favor, due to its active learning, and trounce its GPU-based 'stronger' opponent. Of course, no real surprise here, but I get the feeling most people don't get this basic fact straight, that learning can steadily and significantly increase playing (and analysis) strength.

Yet, I haven't seen much discussion, let alone requests, about letting Leela (or its derivatives) continuing to learn as it plays. Is it that big a deal to change the program so this is made possible? It is puzzling and deflating to see Fat Fritz not being able to continue to learn while a free program can.

OK, as some might say, why should they bother, since they (the Leela team) do this work for free? Well, in that case why won't Chessbase? They surely have the resources to add such a useful feature! In either case, I think it's a poor argument to say that it would be difficult, since a lot more work has gone into prior coding, and adding such a learning feature is a guaranteed instant winner, and worth adding, even with serious effort.

The same thing goes for Komodo, or any other commercial endeavor playing 'catch-up' with stronger engines. Learning automatically and quickly adds strength. It's a simple fact. Can anyone argue with a straight face that they don't care about engine strength? This is what Komodo's programmers imply when they claim that there is just "not enough interest" in this feature. Not enough interest in far greater engine strength?! Cue laugh track, please...

Anyway, this post is not really directed at Komodo's programmers, but rather at Leela and Fat Fritz, much hyped engines that are based on self-learning, yet they can't learn a lick after you've installed them. If you still aren't convinced that learning works exceedingly well, check out Richard Vida's Critter or even Michael Sherwin's Romichess, both freeware, and the latter even open source.

I found this post shocking and decided to try this out. Fat Fritz on my machine running on a Rtx ate Critter for lunch. After 40 games and 40-0, I decided Critter had got crushed enough. I am not sure what hardware your Fat Fritz ran on but, on my machine, they will never be even close.

[Ras]

That could only help its strength, and not hurt it, unless I'm missing something.

What you're missing is even basic understanding on how neural networks actually work. They do not learn lines, they do not learn specific positions. If you tried to make them "learn" with that little knowledge, the only consequence would be that you would make them weaker, not stronger.

[Tony P.]

Strictly speaking, the so-called active learning would only be possible during the training or private testing, not during official competitions, because it supposes that the learner can choose which positions the teacher should label, which can't be forced on the opponent during a competitive game.

I do think NN engines could be adapted to meta-learn online during a tournament as well, but because of the extra computation that it requires, for it to pay off Elo-wise, the tournament has to be much longer than the current best known ones, in order to get a non-negligible sample on the opponent's style and adapt to it, and the rules should allow the engine to write the learning file onto the disk, which, say, CCRL prohibits expressly. And anyway, the engine would need to start with a solid base (prior) strategy learned in advance and then adapt its metaparameters slowly and carefully to the opponent's style by Bayesian inference (e.g. prune the NN vs an opponent that seems to play rather tactically with high NPS, to increase the engine's own NPS and reduce the blunder probability).

In special scenarios like the TCEC SuFi if the prospective opponent is known for sure, I think it's allowed to submit a version of the engine that's been trained with a specific objective to have the best possible winrate vs that opponent. In this case, the games of the match itself wouldn't help much, as opposed to thousands of test games (or millions at a bullet TC) played beforehand vs that opponent or its previous version.

[carldaman]

@Robert Flesher

I have to run FF on a slow GPU, so it's weaker tactically, and a learning engine can latch on to such weakness and play lines FF does not like. Your fast GPU FF is much stronger and far less susceptible to this effect against a lesser engine. The principle still holds, since even the fastest GPU-based entity is not perfect and can lose games.


@Ras

There are two types of learning being discussed, which can create confusion without defining them. The type that Leela trains on can probably better be called 'training'. The other type is what engines like Critter use, or Jeremy Bernstein's SF fork, using a session file that contains refutations and improvements over lines previously played or analyzed. The two types should have nothing to do with each other, although they can both be referred to as 'learning'.

I see no reason why a program like Leela couldn't use a session file to load into its hash, unless its architecture gets in a way somehow. Maybe someone can better elaborate on this.


@Tony P

The learning I'm referring to, session-file based, would be for the benefit of the regular end user, not for rating lists or official tournaments. In this regard, any engine can benefit from this type of learning, not just Leela.


I just found it ironic that Leela, so famous for learning the game so well on its own, is incapable of learning [of course, this would be a different kind of learning -- see above] any longer once in the hands of the user. This is why I made my original post.

[Leo]

The more I think about it, the more I'm convinced it's quite a bit of a travesty that these new self-learning programs actually do not support live learning!

I've recently run a match between the freeware Critter 1.6a, with its session file learning switched on, and the much-vaunted Fat Fritz, a commercial program I had to pay for, and it took less that 10 games for Critter to turn the match score around in its favor, due to its active learning, and trounce its GPU-based 'stronger' opponent. Of course, no real surprise here, but I get the feeling most people don't get this basic fact straight, that learning can steadily and significantly increase playing (and analysis) strength.

Yet, I haven't seen much discussion, let alone requests, about letting Leela (or its derivatives) continuing to learn as it plays. Is it that big a deal to change the program so this is made possible? It is puzzling and deflating to see Fat Fritz not being able to continue to learn while a free program can.

OK, as some might say, why should they bother, since they (the Leela team) do this work for free? Well, in that case why won't Chessbase? They surely have the resources to add such a useful feature! In either case, I think it's a poor argument to say that it would be difficult, since a lot more work has gone into prior coding, and adding such a learning feature is a guaranteed instant winner, and worth adding, even with serious effort.

The same thing goes for Komodo, or any other commercial endeavor playing 'catch-up' with stronger engines. Learning automatically and quickly adds strength. It's a simple fact. Can anyone argue with a straight face that they don't care about engine strength? This is what Komodo's programmers imply when they claim that there is just "not enough interest" in this feature. Not enough interest in far greater engine strength?! Cue laugh track, please...

Anyway, this post is not really directed at Komodo's programmers, but rather at Leela and Fat Fritz, much hyped engines that are based on self-learning, yet they can't learn a lick after you've installed them. If you still aren't convinced that learning works exceedingly well, check out Richard Vida's Critter or even Michael Sherwin's Romichess, both freeware, and the latter even open source.

I found this post shocking and decided to try this out. Fat Fritz on my machine running on a Rtx ate Critter for lunch. After 40 games and 40-0, I decided Critter had got crushed enough. I am not sure what hardware your Fat Fritz ran on but, on my machine, they will never be even close.

I couldn't get Critter to win either against Brainfish.