Page 2 of 4

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 8:37 pm
by Raphexon
Just NN eval definitely isn't an opening book.


But what about policy head?
I definitely consider it bookish, Leela being very deterministic also supports it.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 8:37 pm
by dkappe
dkappe wrote:
Tue Aug 04, 2020 7:44 pm
Oh boy. Most nnue are trained at the beginning without regard to game outcome, often at depth 8. Most of the positions they see are maybe 18 to 19 ply into the game and later. They are in essence an approximation of an engine eval at d8.

You can look at my Toga and Night Nurse (based on Bad Gyal) nets and compare them to each other and the many stockfish nets, then explain to me how they are memorizing openings.
The Toga net in particular starts every game with a 19 ply random walk, at which point it revs up the engine. Every so often you’ll have enough noodling to get something like a KID, but that’s pure chance.

Why does every file large enough to contain an opening book have to a priori be an opening book?

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 8:38 pm
by dkappe
Raphexon wrote:
Tue Aug 04, 2020 8:37 pm
Just NN eval definitely isn't an opening book.


But what about policy head?
I definitely consider it bookish, Leela being very deterministic also supports it.
NNUE just outputs a single eval. No policy.

Also:
https://github.com/dkappe/leela-chess-w ... k-nonsense

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 8:54 pm
by Raphexon
dkappe wrote:
Tue Aug 04, 2020 8:38 pm
Raphexon wrote:
Tue Aug 04, 2020 8:37 pm
Just NN eval definitely isn't an opening book.


But what about policy head?
I definitely consider it bookish, Leela being very deterministic also supports it.
NNUE just outputs a single eval. No policy.

Also:
https://github.com/dkappe/leela-chess-w ... k-nonsense
I wasn't talking NNUE when talking policyhead.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:09 pm
by Twipply
Is this meant to be a fair analysis of the idea presented?

"Leela is an opening book nonsense" The title you chose is far from impartial, which I can't say I like. It makes you sound like you've come into this already thinking you know what the answer is. If I think your analysis is bias or insulting, I'm likely not going to be swayed by it even if you happen to be right.

"I decided to test this opening book hypothesis with one of my favorite nets." What network is your favourite has absolutely no importance here. You should have done an analysis using as many networks as you could, and compared them to a variety of traditional engines, which you didn't do at all.

"I investigated a few opening lines" Looking at three openings is nowhere near comprehensive enough to draw a meaningful conclusion from.

"Good evidence that this net has memorized opening moves?" Not even remotely.

"Not so fast." You're putting words into the mouths of the readers, don't do that - especially not when you're wrong about what they're thinking.

"This is Little Ender" You managed to write a topic about Leela and then used an entirely different network in your attempt at an analysis.


You haven't shown that Leela doesn't contain an opening book, and you haven't shown that engines can't memorise openings.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:19 pm
by Milos
corres wrote:
Tue Aug 04, 2020 6:21 pm
Gian-Carlo Pascutto wrote:
Tue Aug 04, 2020 6:03 pm
...
Some tournaments disallow "books" but allow "neural networks", even if this distinction does not exist in reality, because you can train a neural network to remember openings.
But the most of chess engine can not "read" neural net, so they need common opening book.
So I encode the book for a classical engine in a proprietary format and call it a network. What's the difference?
Since A0 appeared I called it cheating that some engines can use a network of hundreds of MB in size while others can't use their books. It's rather silly.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:28 pm
by Milos
dkappe wrote:
Tue Aug 04, 2020 7:44 pm
Oh boy. Most nnue are trained at the beginning without regard to game outcome, often at depth 8. Most of the positions they see are maybe 18 to 19 ply into the game and later. They are in essence an approximation of an engine eval at d8.
And how is that different to Cerebellum book of Brainfish?
It's a "network" that gives you best move "trained" on SF engine eval of a very high depth. The only real difference to NNUE is that in Cerrebellum the search portion that SF performs is actually also encoded in the book.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:32 pm
by dkappe
Twipply wrote:
Tue Aug 04, 2020 9:09 pm
Is this meant to be a fair analysis of the idea presented?
I’m sorry I hurt your feelings. :D (BTW, I found your engine to be an excellent sparring partner during the development of a0lite.)


It was entirely predictable that I would try my 18 piece or fewer approach with nnue’s.

Here is an nnue that has never seen a position with more than 18 pieces playing against itself, no book. I’m not sure it will be better at endgames, like ender. But it seems to know it’s way around the Spanish.


Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:40 pm
by Milos
dkappe wrote:
Tue Aug 04, 2020 9:32 pm
Twipply wrote:
Tue Aug 04, 2020 9:09 pm
Is this meant to be a fair analysis of the idea presented?
I’m sorry I hurt your feelings. :D (BTW, I found your engine to be an excellent sparring partner during the development of a0lite.)


It was entirely predictable that I would try my 18 piece or fewer approach with nnue’s.

Here is an nnue that has never seen a position with more than 18 pieces playing against itself, no book. I’m not sure it will be better at endgames, like ender. But it seems to know it’s way around the Spanish.

That proves absolutely nothing. SF with eval consisting only of material+mobility+PST also knows its way around spanish and hasn't seen any chess positon ever. That's thanks to search not thanks to eval.
How bad is NNUE eval really one sees when testing SF depth 5 with minimum eval (material+mobility+PST) vs SF-NNUE depth 1. It's a slaughter house.

Re: Is this SF NN almost like 20 MB book?

Posted: Tue Aug 04, 2020 9:48 pm
by Twipply
dkappe wrote:
Tue Aug 04, 2020 9:32 pm
I’m sorry I hurt your feelings. :D
I reacted strongly not because of feelings, but because I think this topic has basically invalidated some of the more recent TCEC Superfinal results, and the admins there should stop ignoring it. However, even if my feelings were hurt, that would not invalidate what I've said nor would it validate your post.
dkappe wrote:
Tue Aug 04, 2020 9:32 pm
(BTW, I found your engine to be an excellent sparring partner during the development of a0lite.)
Thanks. I'm glad it worked well for you.