LCZero Elo

[peter]

Hi Jacob!

Give me any engine in the 2300s-2400s on CCRL (really could go much higher than that, even), have them play without an opening book at a fast time control, and I'd expect to find lines they're losing by move 15

Well, maybe, but the difference is, a "conventional" engine can and may use a book, Leela at the moment "must not" to stay fully "Zero".

Now we could compare bookless Leela to any engine playing with book, couldn't we?

Why then shouldn't we let any engine play with anti- LeelaChessZero- book against LCZ?

It would be another one legal comparison I'd say Celo- wise, just one more way to count (Computer-) "Elo" (Celo) from engine- engine- matches.


Humans neither do play without "books", do they?

So a human player should be allowed to use his personal preparation against bookless LCZ too of course.

If it continues to play not only such single poor lines but keeps playing them so determinately, real Elo are still another one measurement than bookless Celo against other bookless engines is, that's my point.

[MonteCarlo]

I didn't say anything about whether something should or should not be allowed, so all that's a bit irrelevant; anyone is of course free to play matches in whatever conditions they'd like

My point was that it is unsurprising that a 2300ish CCRL engine at fast time controls plays some bad opening lines on its own, nothing more, nothing less

Also, it's only the case that LC0 can't use a book in self-play and training to stay zero; nothing at all other than dev time stops from them adding opening book support as a UCI option for play outside the training pipeline.

[peter]

I didn't say anything about whether something should or should not be allowed, so all that's a bit irrelevant; anyone is of course free to play matches in whatever conditions they'd like

Of course, and the bookless match isn't the "normal" one in "normal" rating list- matches, is it?

My point was that it is unsurprising that an unassisted 2300ish CCRL engine at fast time controls plays some bad opening lines, nothing more, nothing less

And that's the reason, why weak engines even more then strong ones are "normally" used with using books.

Also, it's only the case that LC0 can't use a book in self-play and training to stay zero; nothing at all other than dev time stops from them adding opening book support as a UCI option for play outside the training pipeline.

So if it isn't incompatible with your definition of pure self-learning AI, we have the same povs here anyhow.

Mine would go even further and i'd let it learn against other engines playing with books too and from opening- databases directly, the way Matthew Lai did with Giraffe.

[MonteCarlo]

In normal rating lists, a common book is used to generate diverse starting positions; engines don't use proprietary own books or anything like that.

LC0 can do that just fine, so I'm not sure what the issue is there

Again, at any rate, literally the only point of my post was that it's completely unsurprising for an engine of that strength to misplay positions badly, nothing more, nothing less.

I'm not sure how all these tangents started

[peter]

I'm not sure how all these tangents started

Even better then, so I don't have to say forget it at all