Neural nets, nodes to features?

[Dann Corbit]

Has anyone figured out how to decode the neural nets created by enormous compute effort to turn them back into features and scores/weights?

[chrisw]

Has anyone figured out how to decode the neural nets created by enormous compute effort to turn them back into features and scores/weights?

yes

[towforce]

My personal experience (based on "a bit of playing") is that above about 10 neurons, it becomes prohibitively difficult to work out what's going on in an NN.

I fully accept that somebody doing this seriously would get to a much higher neuron limit than I did.

[smatovic]

...IIRC Deepmind used heat maps to visualize CNN stuff with AlphaGo.

--
Srdja

[chrisw]

...IIRC Deepmind used heat maps to visualize CNN stuff with AlphaGo.

--
Srdja

That's not what was being asked for though

[smatovic]

...IIRC Deepmind used heat maps to visualize CNN stuff with AlphaGo.

--
Srdja

That's not what was being asked for though

Come on, full disclosure

--
Srda

[chrisw]

...IIRC Deepmind used heat maps to visualize CNN stuff with AlphaGo.

--
Srdja

That's not what was being asked for though

Come on, full disclosure

--
Srda

It’s pretty simple. If I gave you 100M Stockfish 15, d7, say, labelled epds, could you extract the virtual SF Piece Values, or the virtual SF PSQT tables from those epds? Well, of course you could, it’s just a linear regression problem, or Texel tune for that matter.

I did this as an experiment several months ago. Only bothering to pull out virtual piece values, but across 8 games phases. And not just for SF, but for as many NNUE engines as were available.
Plot piece value, per piece type against phase and you get a bunch of pretty graphs (5 pieces x 8 phases = 40 graphs per engine). A visual compare of the graph profiles will tell you which engines are using the same or similar training sets, which was my idea of doing this in the first place.

[smatovic]

Nice, thx

--
Srdja