Stockfish NN release (NNUE)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Raphexon
Posts: 216
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Stockfish NN release (NNUE)

Post by Raphexon » Sat Jun 20, 2020 4:43 pm

cdani wrote:
Sat Jun 20, 2020 1:19 pm
Can I take a net and improve it with a new "traninig phase" with new generated training data? I have the evalsave folder with the first training.
Thanks.
If you want to use the nn that appeared in the evalsave folder simply put that nn in the eval folder.
Then you can open the blas binary and use the gensfen command on that.
Then it will generate training data with the NN eval.

User avatar
cdani
Posts: 2134
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Stockfish NN release (NNUE)

Post by cdani » Sat Jun 20, 2020 8:14 pm

Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.

Raphexon
Posts: 216
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Stockfish NN release (NNUE)

Post by Raphexon » Sat Jun 20, 2020 9:29 pm

cdani wrote:
Sat Jun 20, 2020 8:14 pm
Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.
Are you using the blas version?

User avatar
cdani
Posts: 2134
Joined: Sat Jan 18, 2014 9:24 am
Location: Andorra
Contact:

Re: Stockfish NN release (NNUE)

Post by cdani » Sat Jun 20, 2020 10:02 pm

Raphexon wrote:
Sat Jun 20, 2020 9:29 pm
cdani wrote:
Sat Jun 20, 2020 8:14 pm
Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.
Are you using the blas version?
Yes, the one on the readme.txt
stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe

zenpawn
Posts: 334
Joined: Sat Aug 06, 2016 6:31 pm
Location: United States

Re: Stockfish NN release (NNUE)

Post by zenpawn » Sat Jun 20, 2020 10:29 pm

Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!
Erin Dame
Author of RookieMonster

Raphexon
Posts: 216
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Stockfish NN release (NNUE)

Post by Raphexon » Sat Jun 20, 2020 10:42 pm

cdani wrote:
Sat Jun 20, 2020 10:02 pm
Raphexon wrote:
Sat Jun 20, 2020 9:29 pm
cdani wrote:
Sat Jun 20, 2020 8:14 pm
Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.
Are you using the blas version?
Yes, the one on the readme.txt
stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe
Well, thread usage does depend on the phase.
So could be that. I've noticed it doesn't use 100% of the core all the time.
Otherwise that's a fairly strange issue.

pohl4711
Posts: 1272
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Stockfish NN release (NNUE)

Post by pohl4711 » Sun Jun 21, 2020 3:25 am

I need a non-bmi2 compile (but with popc), for AMD Ryzen. BMI2 is terrible slow on AMD

Rom77
Posts: 32
Joined: Wed Oct 24, 2018 5:37 am
Full name: Roman Zhukov

Re: Stockfish NN release (NNUE)

Post by Rom77 » Sun Jun 21, 2020 3:55 am

zenpawn wrote:
Sat Jun 20, 2020 10:29 pm
Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!
I am also trying to understand the structure of NNUE and wrote a message about this a few days ago:
viewtopic.php?f=2&t=74059&start=130#p847616

There is a general scheme and several links by which you can understand the general structure of this neural network. I don’t know if this is what you are asking?

Here is another link:
http://yaneuraou.yaneu.com/2020/06/19/s ... ete-guide/

zenpawn
Posts: 334
Joined: Sat Aug 06, 2016 6:31 pm
Location: United States

Re: Stockfish NN release (NNUE)

Post by zenpawn » Sun Jun 21, 2020 12:24 pm

Rom77 wrote:
Sun Jun 21, 2020 3:55 am
zenpawn wrote:
Sat Jun 20, 2020 10:29 pm
Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!
I am also trying to understand the structure of NNUE and wrote a message about this a few days ago:
viewtopic.php?f=2&t=74059&start=130#p847616

There is a general scheme and several links by which you can understand the general structure of this neural network. I don’t know if this is what you are asking?

Here is another link:
http://yaneuraou.yaneu.com/2020/06/19/s ... ete-guide/
Thanks. In particular, I'm wondering what makes it "efficiently updatable" in a way that, say, LCZero is not.
Erin Dame
Author of RookieMonster

Rom77
Posts: 32
Joined: Wed Oct 24, 2018 5:37 am
Full name: Roman Zhukov

Re: Stockfish NN release (NNUE)

Post by Rom77 » Sun Jun 21, 2020 2:30 pm

zenpawn wrote:
Sun Jun 21, 2020 12:24 pm
Thanks. In particular, I'm wondering what makes it "efficiently updatable" in a way that, say, LCZero is not.
Efficiency is promoted by a special structure of a neural network.

Almost all the weights of the neural network are located on the input layer, and much less in the intermediate layers (whereas Leela has a lot of calculations in the intermediate layers).

Most of the input features are zero. Accordingly, the weights by which they are multiplied in the input layer do not need to be calculated.

When a move is made, the state of only two input features changes (except when the king moves). Therefore, you only need to subtract the weight of the old position of the piece and add a new one. All intermediate layers will need to be recalculated, but there will be few calculations.

Post Reply