Evidence That NNs Work Best With Multiple Modules

[dkappe]

Went through the slides (https://neurips.cc/media/neurips-2021/Slides/26740.pdf). I agree with Milos, this is useless for chess.

These concepts are so high level and general that I don’t think you can make such a blanket statement.

[towforce]

We know what chess computers are doing: generating a list (or tree) of legal moves, and evaluating each one.

A strong human player will look at a position, and as long as the position is reasonably "normal", they will be able to tell you in seconds what the two or three most important things to watch for are.

How does a human do that?

It's something to do with the small (in comparison to LC0 nets, which train on billions of positions) number of positions they've looked at in great detail. They're learning good and bad ways to progress, not to do static evaluations.

So maybe what we should be doing is working out how to generate this data in a form which NNs can learn this skill for a smaller number of positions rather than just sending them a huge database of position/evaluation pairs and saying, "Here NN - do something with that!"

[Sopel]

Went through the slides (https://neurips.cc/media/neurips-2021/Slides/26740.pdf). I agree with Milos, this is useless for chess.

These concepts are so high level and general that I don’t think you can make such a blanket statement.

At some point "high level" and "general" becomes synonymous to "useless".

[Sopel]

We know what chess computers are doing: generating a list (or tree) of legal moves, and evaluating each one.

A strong human player will look at a position, and as long as the position is reasonably "normal", they will be able to tell you in seconds what the two or three most important things to watch for are.

How does a human do that?

It's something to do with the small (in comparison to LC0 nets, which train on billions of positions) number of positions they've looked at in great detail. They're learning good and bad ways to progress, not to do static evaluations.

So maybe what we should be doing is working out how to generate this data in a form which NNs can learn this skill for a smaller number of positions rather than just sending them a huge database of position/evaluation pairs and saying, "Here NN - do something with that!"

You're free to invent neural networks that can do that.

btw. you might find this interesting https://www.researchgate.net/publicatio ... l_Networks

[dkappe]

Went through the slides (https://neurips.cc/media/neurips-2021/Slides/26740.pdf). I agree with Milos, this is useless for chess.

These concepts are so high level and general that I don’t think you can make such a blanket statement.

At some point "high level" and "general" becomes synonymous to "useless".

Category theory is high level and general. When applied properly you get things like Algebraic Topology. You’re complaining that a hammer is not a house. I’m merely pointing out that it can be used to build one.

[towforce]

We know what chess computers are doing: generating a list (or tree) of legal moves, and evaluating each one.

A strong human player will look at a position, and as long as the position is reasonably "normal", they will be able to tell you in seconds what the two or three most important things to watch for are.

How does a human do that?

It's something to do with the small (in comparison to LC0 nets, which train on billions of positions) number of positions they've looked at in great detail. They're learning good and bad ways to progress, not to do static evaluations.

So maybe what we should be doing is working out how to generate this data in a form which NNs can learn this skill for a smaller number of positions rather than just sending them a huge database of position/evaluation pairs and saying, "Here NN - do something with that!"

You're free to invent neural networks that can do that.

I was thinking more in terms of how the data is presented to the NN: my concept is to explicitly teach the NN what's important in a position, rather than show it billions of position/evaluation pairs and hope that it works this out for itself.

btw. you might find this interesting https://www.researchgate.net/publicatio ... l_Networks

Looks interesting: I will read it when I get time.