NNUE nets with depth of 20?

[bastiball]

I always saw that there are 12 or 8 depth that is been used

[AndrewGrant]

I always saw that there are 12 or 8 depth that is been used

Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.

[bastiball]

I always saw that there are 12 or 8 depth that is been used

Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.

I always thought if a Net with 20 depth is much stronger than a net with 8 depth. But it does make sense that some depth is better than the another.

[abgursu]

Nobody remembers we have BrainLearn.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.

[Raphexon]

It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.

[bastiball]

It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.

Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?

[Raphexon]

It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.

Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?

With the architecture used by Stockfish, yes and it's not even close.

[DrCliche]

There's also a sense in which you usually don't want to store knowledge in the network if it's fairly likely to be quickly and correctly elucidated by search. Without a doubt picking an arbitrary constant depth isn't correct, and evaluation depth on training positions should be dynamically adjusted in a manner broadly similar to the way KataGo varies rollout nodes during game generation ...

But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.

[AndrewGrant]

It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.

Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?

With the architecture used by Stockfish, yes and it's not even close.

I don't think its that simple. Noobpwn's d20 nets were not small training samples. He unleashed an army to generate the data. At a certain point the eval eclipses your ability to materialize the gains you see on the board.