this will be the merge of a lifetime : SF 80 Elo+

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Jouni
Posts: 2241
Joined: Wed Mar 08, 2006 7:15 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Jouni » Wed Aug 05, 2020 12:50 pm

Specially when normal SF shows no progress or regression in latest test at https://tests.stockfishchess.org/tests/ ... 64f05e4c4f !
Jouni

User avatar
xr_a_y
Posts: 1337
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by xr_a_y » Wed Aug 05, 2020 12:51 pm

And the code is so clean and not intrusive that it shall be more or less used as it in every other engine.
This would have make sense to release this as a lib (as fathom for instance) rather than directly in SF code because this thing will be copy paste quite a lot in the next few weeks I think ...

Alayan
Posts: 489
Joined: Tue Nov 19, 2019 7:48 pm
Full name: Alayan Feh

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Alayan » Wed Aug 05, 2020 1:18 pm

Jouni wrote:
Wed Aug 05, 2020 12:50 pm
Specially when normal SF shows no progress or regression in latest test at https://tests.stockfishchess.org/tests/ ... 64f05e4c4f !
Only a handful of patches were merged since the previous RT. The difference is well within the noise margin, and the trend is still positive if you account for the SMP RT.

Over short period of times (up to a few months), Stockfish progress is rather irregular. Stockfish had an elo drought after the SF11 release, then a period of very fast progress, now some weeks of slow progress.

Daniel Shawul
Posts: 4065
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Daniel Shawul » Wed Aug 05, 2020 1:21 pm

Gian-Carlo Pascutto wrote:
Tue Aug 04, 2020 6:01 pm
There's still so much room for experimentation. And I suspect that trying and training new approaches is quite a bit faster on a 20M NNUE net compared to a Leela one...

Dream situation for computer chess.
MCTS has for sure tactical problems but do your really think NNUE is ever going to catch up with a 20b net for instance interms of knowledge.
The result of NNUE is impressive so far but I am not so sure if it is the best solution for chess or other games.
What I am very curious about is by how much NNUE improves other engines.

jdart
Posts: 4034
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by jdart » Wed Aug 05, 2020 1:30 pm

Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.

smatovic
Posts: 1610
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by smatovic » Wed Aug 05, 2020 1:44 pm

jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
LC0 vs. NNUE - some tech details...

viewtopic.php?f=2&t=74607

In short:
...
Cos NNUE runs a smaller kind of NN on a CPU efficient it gains more NPS in an
AB search than previous approaches like Giraffe, you can view it in a way that
it can combine both worlds, the LC0 NN part and the SF AB search part, on a CPU.
--
Srdja

User avatar
MikeB
Posts: 4487
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by MikeB » Wed Aug 05, 2020 1:50 pm

jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
Elevator speech - Go programers ( from Japan) took the SF search code and combined it with their GO NN eval and it became quickly one of the best Go engines in the world. They then took their Go engine and converted to a chess NN engine and , got it working , came back to the SF team and saids, "you guys might want take a look at this. it is already playing near the level cur-dev-Stockfish" .. the rest is history as they say .

Caveat - I really do not know the backstory, this is my conjecture from everything I have read so far. The quote above is not real.
Image

Raphexon
Posts: 339
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Raphexon » Wed Aug 05, 2020 2:04 pm

MikeB wrote:
Wed Aug 05, 2020 1:50 pm
jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
Elevator speech - Go programers ( from Japan) took the SF search code and combined it with their GO NN eval and it became quickly one of the best Go engines in the world. They then took their Go engine and converted to a chess NN engine and , got it working , came back to the SF team and saids, "you guys might want take a look at this. it is already playing near the level cur-dev-Stockfish" .. the rest is history as they say .

Caveat - I really do not know the backstory, this is my conjecture from everything I have read so far. The quote above is not real.
Shogi*

And iirc they were already using SF-search before.

Proof of concept SF10 with the 403kb KP net was a few hundred elo stronger than Weiss, not close to SFdev.
Chickenlogic, I and (1-2 weeks later) Jjosh got close to SFdev in about 3-4 weeks of training.
Then Norman (author of Fire) fixed the binaries around 10 july and propelled my 27-6 net from 60 to 100 elo weaker than SF to 30 elo stronger.

That was the moment SF team decided on merging, and when Sergio came in to start training too it really exploded.
Now we think a new net is mediocre when it can't beat SFdev by 100 elo...

User avatar
MikeB
Posts: 4487
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by MikeB » Wed Aug 05, 2020 2:40 pm

Raphexon wrote:
Wed Aug 05, 2020 2:04 pm
MikeB wrote:
Wed Aug 05, 2020 1:50 pm
jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
Elevator speech - Go programers ( from Japan) took the SF search code and combined it with their GO NN eval and it became quickly one of the best Go engines in the world. They then took their Go engine and converted to a chess NN engine and , got it working , came back to the SF team and saids, "you guys might want take a look at this. it is already playing near the level cur-dev-Stockfish" .. the rest is history as they say .

Caveat - I really do not know the backstory, this is my conjecture from everything I have read so far. The quote above is not real.
Shogi*

And iirc they were already using SF-search before.

Proof of concept SF10 with the 403kb KP net was a few hundred elo stronger than Weiss, not close to SFdev.
Chickenlogic, I and (1-2 weeks later) Jjosh got close to SFdev in about 3-4 weeks of training.
Then Norman (author of Fire) fixed the binaries around 10 july and propelled my 27-6 net from 60 to 100 elo weaker than SF to 30 elo stronger.

That was the moment SF team decided on merging, and when Sergio came in to start training too it really exploded.
Now we think a new net is mediocre when it can't beat SFdev by 100 elo...
Thanks - that's the real story!
Image

syzygy
Posts: 4816
Joined: Tue Feb 28, 2012 10:56 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by syzygy » Wed Aug 05, 2020 3:00 pm

jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
Yes, with the NN running on CPU.

Apparently the NN is 20MB but using it in the eval only halves nps, which I find quite remarkable. I haven't tried to understand the code yet, but it seems to rely heavily on vector instructions (unsurprisingly).

So it turns out that NNs are much better than humans at writing evaluation functions (even taking into account speed of execution).

Post Reply