NNUE nets with depth of 20?
Moderators: hgm, Dann Corbit, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
NNUE nets with depth of 20?
I always saw that there are 12 or 8 depth that is been used

 Posts: 940
 Joined: Tue Apr 19, 2016 4:08 am
 Location: U.S.A
 Full name: Andrew Grant
 Contact:
Re: NNUE nets with depth of 20?
Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.
Re: NNUE nets with depth of 20?
I always thought if a Net with 20 depth is much stronger than a net with 8 depth. But it does make sense that some depth is better than the another.AndrewGrant wrote: ↑Thu Jan 14, 2021 3:08 amRemains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.
Re: NNUE nets with depth of 20?
Nobody remembers we have BrainLearn.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.
Re: NNUE nets with depth of 20?
It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 7501000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 7501000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Re: NNUE nets with depth of 20?
Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 8:24 amIt's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 7501000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Re: NNUE nets with depth of 20?
With the architecture used by Stockfish, yes and it's not even close.bastiball wrote: ↑Thu Jan 14, 2021 8:43 amIs it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 8:24 amIt's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 7501000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Re: NNUE nets with depth of 20?
There's also a sense in which you usually don't want to store knowledge in the network if it's fairly likely to be quickly and correctly elucidated by search. Without a doubt picking an arbitrary constant depth isn't correct, and evaluation depth on training positions should be dynamically adjusted in a manner broadly similar to the way KataGo varies rollout nodes during game generation ...
But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.
But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.

 Posts: 940
 Joined: Tue Apr 19, 2016 4:08 am
 Location: U.S.A
 Full name: Andrew Grant
 Contact:
Re: NNUE nets with depth of 20?
I don't think its that simple. Noobpwn's d20 nets were not small training samples. He unleashed an army to generate the data. At a certain point the eval eclipses your ability to materialize the gains you see on the board.Raphexon wrote: ↑Thu Jan 14, 2021 9:07 amWith the architecture used by Stockfish, yes and it's not even close.bastiball wrote: ↑Thu Jan 14, 2021 8:43 amIs it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 8:24 amIt's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 7501000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.