NNUE nets with depth of 20?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
bastiball
Posts: 45
Joined: Tue Oct 20, 2020 2:18 am
Full name: Basti Dangca

NNUE nets with depth of 20?

Post by bastiball » Thu Jan 14, 2021 2:24 am

I always saw that there are 12 or 8 depth that is been used

AndrewGrant
Posts: 1031
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: NNUE nets with depth of 20?

Post by AndrewGrant » Thu Jan 14, 2021 3:08 am

bastiball wrote:
Thu Jan 14, 2021 2:24 am
I always saw that there are 12 or 8 depth that is been used
Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.

bastiball
Posts: 45
Joined: Tue Oct 20, 2020 2:18 am
Full name: Basti Dangca

Re: NNUE nets with depth of 20?

Post by bastiball » Thu Jan 14, 2021 4:47 am

AndrewGrant wrote:
Thu Jan 14, 2021 3:08 am
bastiball wrote:
Thu Jan 14, 2021 2:24 am
I always saw that there are 12 or 8 depth that is been used
Remains to be proven in a meaningful way that some depth is better than another depth.
I believe Boujin tried d20 networks, with very little success.
I always thought if a Net with 20 depth is much stronger than a net with 8 depth. But it does make sense that some depth is better than the another.

abgursu
Posts: 46
Joined: Thu May 14, 2020 1:34 pm
Full name: Ahmet Burak Gursu

Re: NNUE nets with depth of 20?

Post by abgursu » Thu Jan 14, 2021 5:38 am

Nobody remembers we have BrainLearn.
If someone makes a tool for extracting data from experience, we will have a net with much too higher depths.

Raphexon
Posts: 368
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: NNUE nets with depth of 20?

Post by Raphexon » Thu Jan 14, 2021 8:24 am

It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.

bastiball
Posts: 45
Joined: Tue Oct 20, 2020 2:18 am
Full name: Basti Dangca

Re: NNUE nets with depth of 20?

Post by bastiball » Thu Jan 14, 2021 8:43 am

Raphexon wrote:
Thu Jan 14, 2021 8:24 am
It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?

Raphexon
Posts: 368
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: NNUE nets with depth of 20?

Post by Raphexon » Thu Jan 14, 2021 9:07 am

bastiball wrote:
Thu Jan 14, 2021 8:43 am
Raphexon wrote:
Thu Jan 14, 2021 8:24 am
It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?
With the architecture used by Stockfish, yes and it's not even close.

DrCliche
Posts: 21
Joined: Sun Aug 19, 2018 8:57 pm
Full name: Nickolas Reynolds

Re: NNUE nets with depth of 20?

Post by DrCliche » Thu Jan 14, 2021 9:54 am

There's also a sense in which you usually don't want to store knowledge in the network if it's fairly likely to be quickly and correctly elucidated by search. Without a doubt picking an arbitrary constant depth isn't correct, and evaluation depth on training positions should be dynamically adjusted in a manner broadly similar to the way KataGo varies rollout nodes during game generation ...

But that would require some skilled and knowledgeable engineering, and more than a bit of experimentation. As always, talk is easier said than done, the proof is in the pudding, etc.

AndrewGrant
Posts: 1031
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: NNUE nets with depth of 20?

Post by AndrewGrant » Thu Jan 14, 2021 10:18 am

Raphexon wrote:
Thu Jan 14, 2021 9:07 am
bastiball wrote:
Thu Jan 14, 2021 8:43 am
Raphexon wrote:
Thu Jan 14, 2021 8:24 am
It's a quantity vs quality tradeoff.
Required nodes to reach a given depth behaves exponentially.

A billion positions analyzed to depth 8 takes a 24 hours with a modern 12 core.
Depth 12 takes 10 times that.
Depth 20 takes (guesstimate) 750-1000 times as long as depth 8.

Without distributed generation of data everybody is dependent on their own hardware and that makes depth 20 unfeasible on the short term.
Is it a 12 depth net with 500 mil position is much stronger that a 20 depth net with a 2mil position?
With the architecture used by Stockfish, yes and it's not even close.
I don't think its that simple. Noobpwn's d20 nets were not small training samples. He unleashed an army to generate the data. At a certain point the eval eclipses your ability to materialize the gains you see on the board.

Post Reply