Visualization of a NNUE

[shawn]

www.easyzoom.com/image/576437

[smatovic]

Cool

Can you elaborate how it was created and what it shows? I was aware of CNN visualization of Deepmind's A0, heatmaps, didn't know this is possible for NNUE.

--
Srdja

[shawn]

Each one of the 1024 sub-graphs represent a neuron on the hidden layer of the neural network. The color of a square on each sub-graph represents the intensity of the intensity of the weight from a white bishop on that square to that neuron.

Code: https://github.com/xu-shawn/nnue-visualize

[smatovic]

Ah, thanks, so you can iterate and visualize 2 colors * 6 pieces * 1024 hidden layer * 64 squares, but (until now) not a FEN chess position.

Maybe this helps Towforce with his Apophenia

--
Srdja

[shawn]

Pawn - https://www.easyzoom.com/image/576493
Knight - https://www.easyzoom.com/image/576483
Rook - https://www.easyzoom.com/image/576485
Queen - https://www.easyzoom.com/image/576486

Ah, thanks, so you can iterate and visualize 2 colors * 6 pieces * 1024 hidden layer * 64 squares, but (until now) not a FEN chess position.

That's possible and I had done it in the past, but the functionality was removed due to underwhelming results. Just added these back for demonstration purposes:

Kiwipete Weights
https://www.easyzoom.com/image/576488
https://www.easyzoom.com/image/576490

[smatovic]

[..]
That's possible and I had done it in the past, but the functionality was removed due to underwhelming results. Just added these back for demonstration purposes:

Kiwipete Weights
https://www.easyzoom.com/image/576488
https://www.easyzoom.com/image/576490

Nice. When you say it was underwhelming, I assume you were not able to find any patterns correlating with position? For example to take a set of positions with certain schemes and take a look if these repeat in visualization.

[...]

@Towforce: this is really one for you and your elephant hunt

https://www.chessprogramming.org/Perft_ ... Position_2

[d] r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq -

--
Srdja

[towforce]

[..]
That's possible and I had done it in the past, but the functionality was removed due to underwhelming results. Just added these back for demonstration purposes:

Kiwipete Weights
https://www.easyzoom.com/image/576488
https://www.easyzoom.com/image/576490

Nice. When you say it was underwhelming, I assume you were not able to find any patterns correlating with position? For example to take a set of positions with certain schemes and take a look if these repeat in visualization.

[...]

@Towforce: this is really one for you and your elephant hunt

https://www.chessprogramming.org/Perft_ ... Position_2

[d] r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq -

--
Srdja

Hmmm... maybe I need to read more of the thread to understand the context, but I seem to have missed the point here. Could you give me a quick explanation, please?

[smatovic]

Hmmm... maybe I need to read more of the thread to understand the context, but I seem to have missed the point here. Could you give me a quick explanation, please?

It is an visualization of NNUE neural network, you can look and search for patterns, source code by Shawn Xu online:

https://github.com/xu-shawn/nnue-visualize

--
Srdja

[shawn]

Three More Graphs:

1. Visualization of a weak network trained by my own WIP trainer
https://www.easyzoom.com/image/576718

2. Visualization of bishop weights with uniform scaling such that colors from different subplots are comparable (Red = Negative, White = 0, Blue = Positive)
https://www.easyzoom.com/image/576720

3. Similar to 2 except the values are flipped for neurons with negative output weights (STM Perspective)
https://www.easyzoom.com/image/576725

[smatovic]

[...]
2. Visualization of bishop weights with uniform scaling such that colors from different subplots are comparable (Red = Negative, White = 0, Blue = Positive)
https://www.easyzoom.com/image/576720
[...]

Nice color scheme, looks good.

--
Srdja