Stockfish NN release (NNUE)

[Raphexon]

Can I take a net and improve it with a new "traninig phase" with new generated training data? I have the evalsave folder with the first training.
Thanks.

If you want to use the nn that appeared in the evalsave folder simply put that nn in the eval folder.
Then you can open the blas binary and use the gensfen command on that.
Then it will generate training data with the NN eval.

[cdani]

Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.

[Raphexon]

Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.

Are you using the blas version?

[cdani]

Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.

Are you using the blas version?

Yes, the one on the readme.txt
stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe

[zenpawn]

Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!

[Raphexon]

Thanks! Another question.
The training phase uses only one cpu even setting the number of threads. Is this the normal behaviour? In the readme states to set various threads as in the generation phase.

Are you using the blas version?

Yes, the one on the readme.txt
stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe

Well, thread usage does depend on the phase.
So could be that. I've noticed it doesn't use 100% of the core all the time.
Otherwise that's a fairly strange issue.

[pohl4711]

I need a non-bmi2 compile (but with popc), for AMD Ryzen. BMI2 is terrible slow on AMD

[Rom77]

Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!

I am also trying to understand the structure of NNUE and wrote a message about this a few days ago:
http://talkchess.com/forum3/viewtopic.p ... 30#p847616

There is a general scheme and several links by which you can understand the general structure of this neural network. I don’t know if this is what you are asking?

Here is another link:
http://yaneuraou.yaneu.com/2020/06/19/s ... ete-guide/

[zenpawn]

Anyone know of a good resource for understanding NNUE? The shogi paper is behind a paywall and I didn't find much else. Thanks!

I am also trying to understand the structure of NNUE and wrote a message about this a few days ago:
http://talkchess.com/forum3/viewtopic.p ... 30#p847616

There is a general scheme and several links by which you can understand the general structure of this neural network. I don’t know if this is what you are asking?

Here is another link:
http://yaneuraou.yaneu.com/2020/06/19/s ... ete-guide/

Thanks. In particular, I'm wondering what makes it "efficiently updatable" in a way that, say, LCZero is not.

[Rom77]

Thanks. In particular, I'm wondering what makes it "efficiently updatable" in a way that, say, LCZero is not.

Efficiency is promoted by a special structure of a neural network.

Almost all the weights of the neural network are located on the input layer, and much less in the intermediate layers (whereas Leela has a lot of calculations in the intermediate layers).

Most of the input features are zero. Accordingly, the weights by which they are multiplied in the input layer do not need to be calculated.

When a move is made, the state of only two input features changes (except when the king moves). Therefore, you only need to subtract the weight of the old position of the piece and add a new one. All intermediate layers will need to be recalculated, but there will be few calculations.