Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.Rebel wrote: ↑Sat Mar 13, 2021 8:12 pmSolly, ignore previous post, real numbers -
SF12 -> RMS=72.63 | SIM=46.22
SF13 -> RMS=72.20 | SIM=45.65
NNUE Research Project
Moderators: hgm, Rebel, chrisw
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: NNUE Research Project
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 512
- Joined: Tue Sep 29, 2020 4:29 pm
- Location: Dublin, Ireland
- Full name: Madeleine Birchfield
Re: NNUE Research Project
If eval multipliers affect alpha-beta search, then it would be interesting to test the simex of regular Stockfish 13 against Stockfish 13 with a multiplier of 1.5 or 2.dkappe wrote: ↑Sat Mar 13, 2021 8:36 pm Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.
-
- Posts: 1631
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: NNUE Research Project
1.55 was optimal for the Toga engine based on 1+0.6 play. It might be different for SF. I recall Igel used a pretty hefty factor for night nurse (2.5?).Madeleine Birchfield wrote: ↑Sat Mar 13, 2021 8:53 pmIf eval multipliers affect alpha-beta search, then it would be interesting to test the simex of regular Stockfish 13 against Stockfish 13 with a multiplier of 1.5 or 2.dkappe wrote: ↑Sat Mar 13, 2021 8:36 pm Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: NNUE Research Project
Pedone 3Madeleine Birchfield wrote: ↑Sat Mar 13, 2021 4:58 pmThe latest versions of Pedone, Marvin, Komodo, Scorpio, Halogen, Seer, and Tornado should have NNUE as well.
SF12 -> RMS=69.88 | SIM=43.26
SF13 -> RMS=70.18 | SIM=42.74
Halogen 10
SF12 -> RMS=70.54 | SIM=49.25
SF13 -> RMS=71.62 | SIM=49.49
Marvin 5
SF12 -> RMS=86.92 | SIM=48.81
SF13 -> RMS=87.44 | SIM=49.76
Seer 1.2.1
SF12 -> RMS=107.47 | SIM=36.27
SF13 -> RMS=108.57 | SIM=36.69
. Tornado supports nnue, but doesn't have a net of its own.
. I saw no Scorpio nnue executable.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 95
- Joined: Wed Mar 28, 2018 10:16 am
Re: NNUE Research Project
This thread gives help to run Scorpio on NNUE only
http://talkchess.com/forum3/viewtopic.php?f=2&t=76514
Btw: since two days there is a new NNUE Net available on Scorpio Website. Its stored in „nets-nnue.zip“
https://github.com/dshawul/Scorpio/releases - see under Assets
http://talkchess.com/forum3/viewtopic.php?f=2&t=76514
Btw: since two days there is a new NNUE Net available on Scorpio Website. Its stored in „nets-nnue.zip“
https://github.com/dshawul/Scorpio/releases - see under Assets
Last edited by gerhard on Sat Mar 13, 2021 10:59 pm, edited 3 times in total.
-
- Posts: 512
- Joined: Tue Sep 29, 2020 4:29 pm
- Location: Dublin, Ireland
- Full name: Madeleine Birchfield
Re: NNUE Research Project
To me, this plus the Toga III results imply that if one doesn't use the nodchip architecture in Stockfish and doesn't use any Stockfish data in the training process, then one should get a reasonably unique engine; i.e. the Simex is working for NNUE engines as well.Rebel wrote: ↑Sat Mar 13, 2021 9:39 pm Pedone 3
SF12 -> RMS=69.88 | SIM=43.26
SF13 -> RMS=70.18 | SIM=42.74
Halogen 10
SF12 -> RMS=70.54 | SIM=49.25
SF13 -> RMS=71.62 | SIM=49.49
Marvin 5
SF12 -> RMS=86.92 | SIM=48.81
SF13 -> RMS=87.44 | SIM=49.76
Seer 1.2.1
SF12 -> RMS=107.47 | SIM=36.27
SF13 -> RMS=108.57 | SIM=36.69
. Tornado supports nnue, but doesn't have a net of its own.
. I saw no Scorpio nnue executable.
Whether the architecture or the training process is more important in differentiating engines from each other is still up for debate; another Simex test to try would be Minic 3.0 vs Seer 1.2.1; they share the same architecture but have different nets trained on different data by different training processes.
-
- Posts: 1342
- Joined: Wed Mar 08, 2006 9:41 pm
- Location: Morgantown, WV, USA
Re: NNUE Research Project
If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.
https://openchessdb.com/nnue.html
Just added 6.5 Billion more positions.
https://openchessdb.com/nnue.html
Just added 6.5 Billion more positions.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: NNUE Research Project
Thanks for sharing. Do you have info what options were used to generate these bins? What stockfish version?jshriver wrote: ↑Tue Apr 06, 2021 4:14 am If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.
https://openchessdb.com/nnue.html
Just added 6.5 Billion more positions.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: NNUE Research Project
What is a binpack, exactly?jshriver wrote: ↑Tue Apr 06, 2021 4:14 am If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.
https://openchessdb.com/nnue.html
Just added 6.5 Billion more positions.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: NNUE Research Project
A format for training and validation files.Dann Corbit wrote: ↑Wed Apr 07, 2021 7:44 pmWhat is a binpack, exactly?jshriver wrote: ↑Tue Apr 06, 2021 4:14 am If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.
https://openchessdb.com/nnue.html
Just added 6.5 Billion more positions.