Tested several NNUE nets with SF12 and Minic (depth=1) and it seems to me there is a lot a variation between nets, even between the last 7 Sergio nets.
http://rebel13.nl/dump/sf12.html
http://rebel13.nl/dump/minic.html
Maybe this softens Andrew's pain a bit.
What worries me about neural nets (also Lc0) is that it changes your engine playing style while you are not aware of it. Oh wait, that already happens when you only look at the cute-chess results without ever replaying a game
NNUE variation
Moderators: hgm, Rebel, chrisw
-
- Posts: 6994
- Joined: Thu Aug 18, 2011 12:04 pm
NNUE variation
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 476
- Joined: Sun Mar 17, 2019 12:00 pm
- Full name: Henk Drost
Re: NNUE variation
If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
-
- Posts: 4607
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: NNUE variation
Maybe depth=1 similarity simply doesn't work as good with nnue...Raphexon wrote: ↑Tue Sep 29, 2020 11:38 am If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
There should be more tests with various depths and times too for getting a clearer picture regarding the simtest.
https://rwbc-chess.de
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
-
- Posts: 6994
- Joined: Thu Aug 18, 2011 12:04 pm
Re: NNUE variation
What do you suggest?Guenther wrote: ↑Tue Sep 29, 2020 12:03 pmMaybe depth=1 similarity simply doesn't work as good with nnue...Raphexon wrote: ↑Tue Sep 29, 2020 11:38 am If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
There should be more tests with various depths and times too for getting a clearer picture regarding the simtest.
While Fire 7.1 scores 78% at depth=1, it is below 60% at 100ms.
It isn't simple.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4607
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: NNUE variation
? I did not know Fire 7.1 is nnue...Rebel wrote: ↑Tue Sep 29, 2020 8:21 pmWhat do you suggest?Guenther wrote: ↑Tue Sep 29, 2020 12:03 pmMaybe depth=1 similarity simply doesn't work as good with nnue...Raphexon wrote: ↑Tue Sep 29, 2020 11:38 am If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
There should be more tests with various depths and times too for getting a clearer picture regarding the simtest.
While Fire 7.1 scores 78% at depth=1, it is below 60% at 100ms.
It isn't simple.
We are in a nnue thread, so what has Fire to do here?
Ed, please read again what I wrote, especially sentence one, which is the base for sentence two.
https://rwbc-chess.de
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
-
- Posts: 6994
- Joined: Thu Aug 18, 2011 12:04 pm
Re: NNUE variation
Ok, more clear, when search moves in (note the Fire example) the same will happen with NNUE, huge swings in similarity and the longer the time control the more similar engines become. But I already made a start with 100ms, 250, 500, 1000ms and maybe even 4000ms to see if my prediction for NNUE is also true.Guenther wrote: ↑Tue Sep 29, 2020 8:24 pm? I did not know Fire 7.1 is nnue...Rebel wrote: ↑Tue Sep 29, 2020 8:21 pmWhat do you suggest?Guenther wrote: ↑Tue Sep 29, 2020 12:03 pmMaybe depth=1 similarity simply doesn't work as good with nnue...Raphexon wrote: ↑Tue Sep 29, 2020 11:38 am If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
There should be more tests with various depths and times too for getting a clearer picture regarding the simtest.
While Fire 7.1 scores 78% at depth=1, it is below 60% at 100ms.
It isn't simple.
We are in a nnue thread, so what has Fire to do here?
Ed, please read again what I wrote, especially sentence one, which is the base for sentence two.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4607
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: NNUE variation
I wouldn't even test so long tcs, just a few clock cycles (e.g. 16ms rounded up) so 20 40 50 and what ever is slightly above N= X/16Rebel wrote: ↑Tue Sep 29, 2020 8:34 pmOk, more clear, when search moves in (note the Fire example) the same will happen with NNUE, huge swings in similarity and the longer the time control the more similar engines become. But I already made a start with 100ms, 250, 500, 1000ms and maybe even 4000ms to see if my prediction for NNUE is also true.Guenther wrote: ↑Tue Sep 29, 2020 8:24 pm? I did not know Fire 7.1 is nnue...Rebel wrote: ↑Tue Sep 29, 2020 8:21 pmWhat do you suggest?Guenther wrote: ↑Tue Sep 29, 2020 12:03 pmMaybe depth=1 similarity simply doesn't work as good with nnue...Raphexon wrote: ↑Tue Sep 29, 2020 11:38 am If anything that shows that you can just take a network, add some very minor modifications and pass the similarity test with ease.
Could be interesting to test Vondele's net since that's a Sergio net with the last layer SPSA tuned.
https://tests.stockfishchess.org/nns
There should be more tests with various depths and times too for getting a clearer picture regarding the simtest.
While Fire 7.1 scores 78% at depth=1, it is below 60% at 100ms.
It isn't simple.
We are in a nnue thread, so what has Fire to do here?
Ed, please read again what I wrote, especially sentence one, which is the base for sentence two.
and depths 2-12 or so. (not every depth needed)
https://rwbc-chess.de
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
-
- Posts: 216
- Joined: Sun Jan 22, 2017 8:30 pm
- Location: Russia
Re: NNUE variation
Ed, you wrote yourself in the 2019 similarity report:
To police CPU NN origins, you'll need to lower the thresholds. From the user perspective, though, I'm just happy to see the bigger variety regardless of the baseline.All tested engines in this report are of the alpha-beta type, so our proposed baseline is an alpha-beta baseline. When we test as many neural net engines as possible for our next report, we may well discover a different baseline figure for move variance, since neural net engines anecdotally evaluate positions differently to alpha-beta handcrafted evaluation functions.
-
- Posts: 6994
- Joined: Thu Aug 18, 2011 12:04 pm
Re: NNUE variation
Here are some results -
http://rebel13.nl/dump/nnue-depth-1.html
http://rebel13.nl/dump/nnue-100ms.html
http://rebel13.nl/dump/nnue-250ms.html
http://rebel13.nl/dump/nnue-500ms.html
No serious similarity in sight, but rising due to increasing time control (strength), same pattern as pure AB engines.
http://rebel13.nl/dump/nnue-depth-1.html
http://rebel13.nl/dump/nnue-100ms.html
http://rebel13.nl/dump/nnue-250ms.html
http://rebel13.nl/dump/nnue-500ms.html
No serious similarity in sight, but rising due to increasing time control (strength), same pattern as pure AB engines.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 6994
- Joined: Thu Aug 18, 2011 12:04 pm
Re: NNUE variation
Added 1000ms and 2000ms
http://rebel13.nl/dump/nnue-1000ms.html
http://rebel13.nl/dump/nnue-2000ms.html
Same pattern, nothing to worry about.
NNUE doesn't change the diversity.
http://rebel13.nl/dump/nnue-1000ms.html
http://rebel13.nl/dump/nnue-2000ms.html
Same pattern, nothing to worry about.
NNUE doesn't change the diversity.
90% of coding is debugging, the other 10% is writing bugs.