Asking to people who believe Leela NN is a book, what they think about SF NN now?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jhellis3
Posts: 485
Joined: Fri Aug 16, 2013 10:36 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by jhellis3 » Sun Sep 13, 2020 11:44 pm

dkappe wrote: You remind me of that guy who prattles on and on about Hopfield networks, believing that just because they have “neural network” in their name, they have anything to do with ResNets.
dkappe wrote:You *are* the Hopfield Networks guy. :lol: :lol:

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by dkappe » Sun Sep 13, 2020 11:48 pm

jhellis3 wrote:
Sun Sep 13, 2020 11:44 pm
dkappe wrote: You remind me of that guy who prattles on and on about Hopfield networks, believing that just because they have “neural network” in their name, they have anything to do with ResNets.
dkappe wrote:You *are* the Hopfield Networks guy. :lol: :lol:
Perfectly fine. I was attacking your knowledge and understanding, not your motive or character.

jhellis3
Posts: 485
Joined: Fri Aug 16, 2013 10:36 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by jhellis3 » Sun Sep 13, 2020 11:49 pm

Lol? WTF?

Well, you did make me laugh anyway.... I'll give you that.

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by dkappe » Sun Sep 13, 2020 11:56 pm

jhellis3 wrote:
Sun Sep 13, 2020 11:49 pm
Lol? WTF?

Well, you did make me laugh anyway.... I'll give you that.
It seems that basic set theory reasoning and ad hominem are the only places you can detect a fallacy, so you shoehorn everything into them, whether it fits or not. :lol:

jhellis3
Posts: 485
Joined: Fri Aug 16, 2013 10:36 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by jhellis3 » Sun Sep 13, 2020 11:58 pm

Bruh, you need learn to quit while you are behind... Jesus Wept.

Milos
Posts: 3832
Joined: Wed Nov 25, 2009 12:47 am

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Milos » Mon Sep 14, 2020 2:43 am

dkappe wrote:
Sun Sep 13, 2020 9:54 pm
Somehow a leela style net that never saw positions with more than 18 pieces was able to “memorize” opening moves. Yet somehow people arguing this tired old argument are unable to memorize this refutation. :D
You my friend have a conflict with a basic logic. The "fact" that the net with 18 pieces can't memorize the opening with 32 pieces has absolutely nothing to do with refuting that net with 32 pieces can memorize the opening with 32 pieces. You are just repeating your non sequitur argument, nothing else.
To simplify the argument for you so you'd be able to actually follow - NN is equal to book+evaluation. When you enter a position with 32 pieces into NN that is trained on 18 pieces NN will perform only eval. When you enter a position that has 32 pieces and that net has actually been trained at it will output book score adjusted by its eval.

dkappe
Posts: 518
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by dkappe » Mon Sep 14, 2020 3:18 am

Milos wrote:
Mon Sep 14, 2020 2:43 am
dkappe wrote:
Sun Sep 13, 2020 9:54 pm
Somehow a leela style net that never saw positions with more than 18 pieces was able to “memorize” opening moves. Yet somehow people arguing this tired old argument are unable to memorize this refutation. :D
You my friend have a conflict with a basic logic. The "fact" that the net with 18 pieces can't memorize the opening with 32 pieces has absolutely nothing to do with refuting that net with 32 pieces can memorize the opening with 32 pieces. You are just repeating your non sequitur argument, nothing else.
To simplify the argument for you so you'd be able to actually follow - NN is equal to book+evaluation. When you enter a position with 32 pieces into NN that is trained on 18 pieces NN will perform only eval. When you enter a position that has 32 pieces and that net has actually been trained at it will output book score adjusted by its eval.
So your hypothesis is that a leela type network memorizes openings. How do we test this hypothesis? What evidence, for instance, would show this hypothesis to be false? If there is no possible way for the hypothesis to be disproven, then it is vacuous.

So, how would one go about trying to disprove it?

Leo
Posts: 957
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Leo » Mon Sep 14, 2020 3:21 am

Alayan wrote:
Tue Aug 25, 2020 3:30 pm
People who believe Leela networks that see some early-game position hundreds of thousands to tens of millions of times in a training run perform no memorization whatsoever are deluding themselves.

This doesn't mean Leela networks are just a book, their ability in unknown positions proves there is a lot of pattern recognition going on, but perform a moderate start position alteration (invert bishops and knights, keep king/rook the same for castling) and you'll see relative performance drop against an engine with handcrafted eval. You may argue that it's because the patterns that come from the modified start position are somewhat different, but part of it may just as well come from the net weights not being tuned to play the best early moves. That is, not having memorized them.

Some people, like dkappe, argue about this topic as if there can only be two truths, Leela 0% book and Leela 100% book. This is a fallacy.

That the memorization is not some trivial mapping like a list of positions with move/eval is irrelevant. If you analyzed the brains of human chess players, you wouldn't be able to find neurons responsible for knowing that 1. e4 and 1. d4 are the best opening moves. But if you claim human chess players don't memorize early opening moves, you're delusional. So, the ability to memorize isn't determined by the way the memorized data is stored.

To put it another way : Magnus Carlsen isn't an opening book. But Magnus Carlsen do memorize openings, and it makes him play stronger.

Bringing SF-NNUE to the table has little relevance. NNUE doesn't feature a "policy" output that gives a weighted ordered list of expected best moves in a position, it only outputs an evaluation. The core of "Leela is a book" claim comes from the behavior of this "policy" output that can often play the theory move with no search whatsoever.

Besides, if a Leela-like net was paired with SF search, the arguments about "book" behavior would be just as relevant.
I cant explain it like you can but I agree with your assesment.
Advanced Micro Devices fan.

Leo
Posts: 957
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Leo » Mon Sep 14, 2020 3:26 am

Milos wrote:
Wed Aug 26, 2020 2:19 pm
Dann Corbit wrote:
Wed Aug 26, 2020 8:00 am
AndrewGrant wrote:
Wed Aug 26, 2020 5:18 am
Nay Lin Tun wrote:
Tue Aug 25, 2020 4:58 am
As per title.
I've never been a fan of Leela, due to the fact that its just a book.
And frankly, I don't know how to read.
Aye, and there's the rub.
Leela sings a beautiful song, and none of us even knows the key.

But I disagree. There is no book.

The robot bears the blade just as well, and with malice.

The book was not written by the lady
It was just a set of instructions how to kill
We, the gentle folk, knew little of her ways
But she borne razor knives the same, to force her wicked will
Your argumentation is on the level of cavemen. You are attaching divine attributes to things you don't understand. Cavemen thought sun is a beautiful god just because they didn't understand that sun is actually a giant fusion reactor.
That the real God made.
Advanced Micro Devices fan.

cucumber
Posts: 128
Joined: Sun Oct 14, 2018 6:21 pm
Full name: JSmith

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by cucumber » Mon Sep 14, 2020 4:09 am

People who don't think SPSA and SF's parameter-set have tuned SF toward/against certain opening moves are also delusional. You don't need to "memorize" openings to tune an engine into preferring one opening over another. Small eval and search perturbations can substantially influence SF's opening preferences, and SPSA has made huge perturbations. Opening moves have a huge influence on game outcome. If SPSA is half decent, SF is an opening book for the same reasons that Leela is.

Post Reply