Asking to people who believe Leela NN is a book, what they think about SF NN now?

[jhellis3]

You remind me of that guy who prattles on and on about Hopfield networks, believing that just because they have “neural network” in their name, they have anything to do with ResNets.

You *are* the Hopfield Networks guy.

[dkappe]

You remind me of that guy who prattles on and on about Hopfield networks, believing that just because they have “neural network” in their name, they have anything to do with ResNets.

You *are* the Hopfield Networks guy.

Perfectly fine. I was attacking your knowledge and understanding, not your motive or character.

[jhellis3]

Lol? WTF?

Well, you did make me laugh anyway.... I'll give you that.

[dkappe]

Lol? WTF?

Well, you did make me laugh anyway.... I'll give you that.

It seems that basic set theory reasoning and ad hominem are the only places you can detect a fallacy, so you shoehorn everything into them, whether it fits or not.

[jhellis3]

Bruh, you need learn to quit while you are behind... Jesus Wept.

[Milos]

Somehow a leela style net that never saw positions with more than 18 pieces was able to “memorize” opening moves. Yet somehow people arguing this tired old argument are unable to memorize this refutation.

You my friend have a conflict with a basic logic. The "fact" that the net with 18 pieces can't memorize the opening with 32 pieces has absolutely nothing to do with refuting that net with 32 pieces can memorize the opening with 32 pieces. You are just repeating your non sequitur argument, nothing else.
To simplify the argument for you so you'd be able to actually follow - NN is equal to book+evaluation. When you enter a position with 32 pieces into NN that is trained on 18 pieces NN will perform only eval. When you enter a position that has 32 pieces and that net has actually been trained at it will output book score adjusted by its eval.

[dkappe]

Somehow a leela style net that never saw positions with more than 18 pieces was able to “memorize” opening moves. Yet somehow people arguing this tired old argument are unable to memorize this refutation.

You my friend have a conflict with a basic logic. The "fact" that the net with 18 pieces can't memorize the opening with 32 pieces has absolutely nothing to do with refuting that net with 32 pieces can memorize the opening with 32 pieces. You are just repeating your non sequitur argument, nothing else.
To simplify the argument for you so you'd be able to actually follow - NN is equal to book+evaluation. When you enter a position with 32 pieces into NN that is trained on 18 pieces NN will perform only eval. When you enter a position that has 32 pieces and that net has actually been trained at it will output book score adjusted by its eval.

So your hypothesis is that a leela type network memorizes openings. How do we test this hypothesis? What evidence, for instance, would show this hypothesis to be false? If there is no possible way for the hypothesis to be disproven, then it is vacuous.

So, how would one go about trying to disprove it?

[Leo]

People who believe Leela networks that see some early-game position hundreds of thousands to tens of millions of times in a training run perform no memorization whatsoever are deluding themselves.

This doesn't mean Leela networks are just a book, their ability in unknown positions proves there is a lot of pattern recognition going on, but perform a moderate start position alteration (invert bishops and knights, keep king/rook the same for castling) and you'll see relative performance drop against an engine with handcrafted eval. You may argue that it's because the patterns that come from the modified start position are somewhat different, but part of it may just as well come from the net weights not being tuned to play the best early moves. That is, not having memorized them.

Some people, like dkappe, argue about this topic as if there can only be two truths, Leela 0% book and Leela 100% book. This is a fallacy.

That the memorization is not some trivial mapping like a list of positions with move/eval is irrelevant. If you analyzed the brains of human chess players, you wouldn't be able to find neurons responsible for knowing that 1. e4 and 1. d4 are the best opening moves. But if you claim human chess players don't memorize early opening moves, you're delusional. So, the ability to memorize isn't determined by the way the memorized data is stored.

To put it another way : Magnus Carlsen isn't an opening book. But Magnus Carlsen do memorize openings, and it makes him play stronger.

Bringing SF-NNUE to the table has little relevance. NNUE doesn't feature a "policy" output that gives a weighted ordered list of expected best moves in a position, it only outputs an evaluation. The core of "Leela is a book" claim comes from the behavior of this "policy" output that can often play the theory move with no search whatsoever.

Besides, if a Leela-like net was paired with SF search, the arguments about "book" behavior would be just as relevant.

I cant explain it like you can but I agree with your assesment.

[Leo]

As per title.

I've never been a fan of Leela, due to the fact that its just a book.
And frankly, I don't know how to read.

Aye, and there's the rub.
Leela sings a beautiful song, and none of us even knows the key.

But I disagree. There is no book.

The robot bears the blade just as well, and with malice.

The book was not written by the lady
It was just a set of instructions how to kill
We, the gentle folk, knew little of her ways
But she borne razor knives the same, to force her wicked will

Your argumentation is on the level of cavemen. You are attaching divine attributes to things you don't understand. Cavemen thought sun is a beautiful god just because they didn't understand that sun is actually a giant fusion reactor.

That the real God made.

[cucumber]

People who don't think SPSA and SF's parameter-set have tuned SF toward/against certain opening moves are also delusional. You don't need to "memorize" openings to tune an engine into preferring one opening over another. Small eval and search perturbations can substantially influence SF's opening preferences, and SPSA has made huge perturbations. Opening moves have a huge influence on game outcome. If SPSA is half decent, SF is an opening book for the same reasons that Leela is.