nnue evaluation question

[Uri Blass]

I am not sure if I understand correctly what is nnue.

Does it means that it is some black box that humans cannot understand what is in it.
I even do not know where to find the evaluation of stockfish to see if I can understand something and I asked in a different thread where to find it(do not plan to spend a lot of time to search it by looking at different files).

I basically remember from reading claims that every engine can improve the evaluation function it has by NNUE combined with the original evaluation and optimizing it based on search
but I do not understand what is the new evaluation function.

I read the following about stockfish that basically I understand nothing from it:

"The neural network of Stockfish NNUE consists of four layers, W1 through W4. The input layer W1 is heavily overparametrized, feeding in the board representation for various king configurations. The efficiency of the net is due to incremental update of W1 in make and unmake move, where only a fraction of its neurons need to be recalculated. The remaining three layers with 32x2x256, 32x32 and 32x1 weights are computational less expensive, best calculated using appropriate SIMD instructions like AVX2 on x86-64, or if available, AVX-512."

Is there a step by step instruction how to calculate evaluation of one position with nnue for humans?

Stockfish with nnue is not an extremely slow searcher so I think that it should be possible for humans to follow all the calculation even if it takes many hours for a human to calculate one position.

[Dann Corbit]

https://www.researchgate.net/figure/How ... _308094289

You won't find explanations of exactly what the layers are doing as far as chess decisions.
They teach themselves based on the inputs and write the data to a file, but to us it is opaque.

[smatovic]

Hmm, you try to read a neural network like a multidimensional PSQT? It is not that simple, as mentioned, NNs are pretty much a black-box*

Re: How to get started with NNUE
viewtopic.php?p=957108#p957108

* Understanding AlphaZero Neural Network’s SuperHuman Chess Ability
https://www.marktechpost.com/2021/12/16 ... s-ability/

--
Srdja

[Uri Blass]

https://www.researchgate.net/figure/How ... _308094289

You won't find explanations of exactly what the layers are doing as far as chess decisions.
They teach themselves based on the inputs and write the data to a file, but to us it is opaque.

I think that at least it should be possible to have a discription of the mathematical computations that stockfish does to evaluate a single position even if humans do not understand how it is connected to chess.

How many calculation does it do in order to evaluate a single position?

[hgm]

Each link in the NN requires a multiplication and an addition, and the sum S of those for the 'neuron' they lead to requires the calculation of max(0, S). (At least, I though the units of the Stockfish NN are ReLu?)

So basically you have 256 sets of PST for each King location, and to update the 'material evaluation' for a set requires subtraction of the PST value for the moved piece in its old location, and addition of it in the new location. The 256 thus updated material evaluations are fully connected to the next layer of 32 ReLu units. So that is 32*256 additions plus multiplications. The next layer is again 32 units and fully connected, so this time 32*32 additions plus multiplications. Finally, to get the single output you need 32 more multiplications and additions.

(I could be off by a factor 2, because there are 256 PST for the location of one King, and 256 for the location of the other.)

[chesskobra]

You could start with https://github.com/asdfjkl/neural_network_chess (which I have browsed through, not read in detail). I think it is a very good reference, which also shows code to implement NN for some simple games yourself.

[smatovic]

...I guess the non-linear activation function prevents humans from "reading" neural networks weights/layers, or alike.

https://en.wikipedia.org/wiki/Activation_function

https://en.wikipedia.org/wiki/Rectifier ... _networks)

https://en.wikipedia.org/wiki/Softmax_function

--
Srdja

[hgm]

If the activation function was linear, the whole network could be collapsed to a single layer. The output would just be the weighted sum of all inputs.

So basically we cannot understand what it does, because it actually does something.

[Joost Buijs]

It is beyond human understanding, perhaps aliens comprehend what it actually does.

[Henk]

Black boxes (that is AI) taking over the world. The chances of anything coming from .. are a million to one.