Asking to people who believe Leela NN is a book, what they think about SF NN now?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Nay Lin Tun
Posts: 628
Joined: Mon Jan 16, 2012 5:34 am

Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Nay Lin Tun » Tue Aug 25, 2020 4:58 am

As per title.

Dann Corbit
Posts: 11510
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Dann Corbit » Tue Aug 25, 2020 5:13 am

Why would you want the opinions from stupid people?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Tord
Posts: 5
Joined: Tue Feb 27, 2018 10:29 am

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Tord » Tue Aug 25, 2020 6:45 am

That's kind of like asking people who believe watermelons are helicopters what they think about jellyfish. Even assuming that such people exist (and I've never seen the "Leela = book" thing), why would you, I or anyone care about their opinions about jellyfish?

DrCliche
Posts: 13
Joined: Sun Aug 19, 2018 8:57 pm
Full name: Nickolas Reynolds

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by DrCliche » Tue Aug 25, 2020 7:14 am

SFNNUE is a much smaller book.

:D

User avatar
hgm
Posts: 24730
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by hgm » Tue Aug 25, 2020 8:20 am

I suppose it is an attempt to educate them, by ecouraging them to reconsider their arguments.

Alayan
Posts: 389
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Alayan » Tue Aug 25, 2020 3:30 pm

People who believe Leela networks that see some early-game position hundreds of thousands to tens of millions of times in a training run perform no memorization whatsoever are deluding themselves.

This doesn't mean Leela networks are just a book, their ability in unknown positions proves there is a lot of pattern recognition going on, but perform a moderate start position alteration (invert bishops and knights, keep king/rook the same for castling) and you'll see relative performance drop against an engine with handcrafted eval. You may argue that it's because the patterns that come from the modified start position are somewhat different, but part of it may just as well come from the net weights not being tuned to play the best early moves. That is, not having memorized them.

Some people, like dkappe, argue about this topic as if there can only be two truths, Leela 0% book and Leela 100% book. This is a fallacy.

That the memorization is not some trivial mapping like a list of positions with move/eval is irrelevant. If you analyzed the brains of human chess players, you wouldn't be able to find neurons responsible for knowing that 1. e4 and 1. d4 are the best opening moves. But if you claim human chess players don't memorize early opening moves, you're delusional. So, the ability to memorize isn't determined by the way the memorized data is stored.

To put it another way : Magnus Carlsen isn't an opening book. But Magnus Carlsen do memorize openings, and it makes him play stronger.

Bringing SF-NNUE to the table has little relevance. NNUE doesn't feature a "policy" output that gives a weighted ordered list of expected best moves in a position, it only outputs an evaluation. The core of "Leela is a book" claim comes from the behavior of this "policy" output that can often play the theory move with no search whatsoever.

Besides, if a Leela-like net was paired with SF search, the arguments about "book" behavior would be just as relevant.

brianr
Posts: 442
Joined: Thu Mar 09, 2006 2:01 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by brianr » Tue Aug 25, 2020 4:07 pm

The term "book" within the computer chess domain has a very specific meaning, which has been consistent for decades.
That a net can approximate a book in a rather general theoretical way does not make it a book within this context.
How humans memorize things is totally irrelevant.
Accordingly, the mapping is relevant.

Alayan
Posts: 389
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Alayan » Tue Aug 25, 2020 4:51 pm

If a NN is trained to match a 1:1 position/move book in a small finite set of position and manages to produce 100% match, it's a book despite storing things in a different format.

The relevant topic of interest is whether memorization of move choice for specific positions occurs, and how much. The human analogy is very relevant in this regard.

In the end, people who bring arguments like "Leela is a book" do so because they believe that it gets an unfair advantage from memorized positions in tournaments/rating lists where classical 1:1 books are banned (and for good reasons). Arguing about the precise definition of "book" doesn't address the main point of contention, which is how much does the memorization matter.

Jouni
Posts: 2199
Joined: Wed Mar 08, 2006 7:15 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by Jouni » Tue Aug 25, 2020 5:06 pm

Because eval file is 21 MB there is room for about 2 million positions. Not enough to get much boost even if used for opening moves.
Jouni

brianr
Posts: 442
Joined: Thu Mar 09, 2006 2:01 pm

Re: Asking to people who believe Leela NN is a book, what they think about SF NN now?

Post by brianr » Tue Aug 25, 2020 5:10 pm

Alayan wrote:
Tue Aug 25, 2020 4:51 pm
If a NN is trained to match a 1:1 position/move book in a small finite set of position and manages to produce 100% match, it's a book despite storing things in a different format.
Ah, but traditional computer chess "books" are not 1:1 at all.
Of course, there are typically several net policy moves with varying probabilities, but the net will always play the same move, unless the search time is changed. No traditional book implementation that I am aware of (and I have looked at the code for many and written several) uses time. Thus, the net book move depends on the search and time, which are not part of a regular book.

Post Reply