Asking to people who believe Leela NN is a book, what they think about SF NN now?

[Nay Lin Tun]

As per title.

[Dann Corbit]

Why would you want the opinions from stupid people?

[Tord]

That's kind of like asking people who believe watermelons are helicopters what they think about jellyfish. Even assuming that such people exist (and I've never seen the "Leela = book" thing), why would you, I or anyone care about their opinions about jellyfish?

[DrCliche]

SFNNUE is a much smaller book.

[hgm]

I suppose it is an attempt to educate them, by ecouraging them to reconsider their arguments.

[Alayan]

People who believe Leela networks that see some early-game position hundreds of thousands to tens of millions of times in a training run perform no memorization whatsoever are deluding themselves.

This doesn't mean Leela networks are just a book, their ability in unknown positions proves there is a lot of pattern recognition going on, but perform a moderate start position alteration (invert bishops and knights, keep king/rook the same for castling) and you'll see relative performance drop against an engine with handcrafted eval. You may argue that it's because the patterns that come from the modified start position are somewhat different, but part of it may just as well come from the net weights not being tuned to play the best early moves. That is, not having memorized them.

Some people, like dkappe, argue about this topic as if there can only be two truths, Leela 0% book and Leela 100% book. This is a fallacy.

That the memorization is not some trivial mapping like a list of positions with move/eval is irrelevant. If you analyzed the brains of human chess players, you wouldn't be able to find neurons responsible for knowing that 1. e4 and 1. d4 are the best opening moves. But if you claim human chess players don't memorize early opening moves, you're delusional. So, the ability to memorize isn't determined by the way the memorized data is stored.

To put it another way : Magnus Carlsen isn't an opening book. But Magnus Carlsen do memorize openings, and it makes him play stronger.

Bringing SF-NNUE to the table has little relevance. NNUE doesn't feature a "policy" output that gives a weighted ordered list of expected best moves in a position, it only outputs an evaluation. The core of "Leela is a book" claim comes from the behavior of this "policy" output that can often play the theory move with no search whatsoever.

Besides, if a Leela-like net was paired with SF search, the arguments about "book" behavior would be just as relevant.

[brianr]

The term "book" within the computer chess domain has a very specific meaning, which has been consistent for decades.
That a net can approximate a book in a rather general theoretical way does not make it a book within this context.
How humans memorize things is totally irrelevant.
Accordingly, the mapping is relevant.

[Alayan]

If a NN is trained to match a 1:1 position/move book in a small finite set of position and manages to produce 100% match, it's a book despite storing things in a different format.

The relevant topic of interest is whether memorization of move choice for specific positions occurs, and how much. The human analogy is very relevant in this regard.

In the end, people who bring arguments like "Leela is a book" do so because they believe that it gets an unfair advantage from memorized positions in tournaments/rating lists where classical 1:1 books are banned (and for good reasons). Arguing about the precise definition of "book" doesn't address the main point of contention, which is how much does the memorization matter.

[Jouni]

Because eval file is 21 MB there is room for about 2 million positions. Not enough to get much boost even if used for opening moves.

[brianr]

If a NN is trained to match a 1:1 position/move book in a small finite set of position and manages to produce 100% match, it's a book despite storing things in a different format.

Ah, but traditional computer chess "books" are not 1:1 at all.
Of course, there are typically several net policy moves with varying probabilities, but the net will always play the same move, unless the search time is changed. No traditional book implementation that I am aware of (and I have looked at the code for many and written several) uses time. Thus, the net book move depends on the search and time, which are not part of a regular book.