NNUE nets with depth of 20?
Moderators: hgm, Rebel, chrisw
-
- Posts: 5243
- Joined: Tue Oct 20, 2020 4:18 am
- Full name: Basti Dangca
NNUE nets with depth of 20?
I always saw that there are 12 or 8 depth that is been used
Basti Dangca
CCRL testing group
CCRL testing group
-
- Posts: 1759
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: NNUE nets with depth of 20?
Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 5243
- Joined: Tue Oct 20, 2020 4:18 am
- Full name: Basti Dangca
Re: NNUE nets with depth of 20?
I always thought if a Net with 20 depth is much stronger than a net with 8 depth. But it does make sense that some depth is better than the another.AndrewGrant wrote: ↑Thu Jan 14, 2021 4:08 amRemains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.
Basti Dangca
CCRL testing group
CCRL testing group
-
- Posts: 91
- Joined: Thu May 14, 2020 3:34 pm
- Full name: A. B. Gursu
Re: NNUE nets with depth of 20?
Nobody remembers we have BrainLearn.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: NNUE nets with depth of 20?
It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
-
- Posts: 5243
- Joined: Tue Oct 20, 2020 4:18 am
- Full name: Basti Dangca
Re: NNUE nets with depth of 20?
Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 9:24 am It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Basti Dangca
CCRL testing group
CCRL testing group
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: NNUE nets with depth of 20?
With the architecture used by Stockfish, yes and it's not even close.bastiball wrote: ↑Thu Jan 14, 2021 9:43 amIs it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 9:24 am It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
-
- Posts: 65
- Joined: Sun Aug 19, 2018 10:57 pm
- Full name: Nickolas Reynolds
Re: NNUE nets with depth of 20?
There's also a sense in which you usually don't want to store knowledge in the network if it's fairly likely to be quickly and correctly elucidated by search. Without a doubt picking an arbitrary constant depth isn't correct, and evaluation depth on training positions should be dynamically adjusted in a manner broadly similar to the way KataGo varies rollout nodes during game generation ...
But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.
But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.
-
- Posts: 1759
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: NNUE nets with depth of 20?
I don't think its that simple. Noobpwn's d20 nets were not small training samples. He unleashed an army to generate the data. At a certain point the eval eclipses your ability to materialize the gains you see on the board.Raphexon wrote: ↑Thu Jan 14, 2021 10:07 amWith the architecture used by Stockfish, yes and it's not even close.bastiball wrote: ↑Thu Jan 14, 2021 9:43 amIs it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?Raphexon wrote: ↑Thu Jan 14, 2021 9:24 am It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.
A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.
Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )