How to scale stockfish NNUE score?

Discussion of chess software programming and technical issues.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 8:42 am

Hi guys, thanks to amazing library to probe stockfish NNUE by Daniel Shawul even a noob like me can now get NNUE score for each position, but it turns out that using that score directly leads to a VERY STRANGE behavior of my engine = it makes strange moves from time to time and sacrifices the queen for nothing in every game. I feel a bit confused at this point.

My questions are:
1. Do I need to evaluate material if I want to use pure NNUE eval?
2. Do I need to scale NNUE eval?
3. Do I need to tune material weights? (I heard SF uses around 200 for paen value while my engine does around 100)

Actually what should I do with NNUE score to make my engine play normal chess?

Please help.

User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 10:11 am

Well, it seems like I've managed to fix this issue:

1. Scaled NNUE score like in PURE NNUE in CFish

Code: Select all

int score = eval_nnue(side, pieces, squares) * 5 / 4;
2. Assuming SF's relative pawn score is around 200 points (while my engine used around 100 points, depending on game phase) I've multiplied my piece weights by 2:

Code: Select all

// get opening/endgame material score
score_opening += material_score[opening][piece] * 2;
score_endgame += material_score[endgame][piece] * 2;
3. Finally I've added NNUE score to interpolated material score:

Code: Select all

// return final evaluation based on side
score += evaluate_nnue();
This resulted the following game (I used fixed depth 8 to make sure that search is the same while only eval difers):


Can I now say NNUE scores are now scaled correctly?

User avatar
hgm
Posts: 25104
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: How to scale stockfish NNUE score?

Post by hgm » Sat Oct 17, 2020 12:35 pm

Let me get this straight: NNUE is just one term of the Stockfish evaluation, and there are other terms?

Is material the only other term (and is it tapered from opening to end-game)?
Get rid of the shit: vote for SHID!

Daniel Shawul
Posts: 4056
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: How to scale stockfish NNUE score?

Post by Daniel Shawul » Sat Oct 17, 2020 3:27 pm

maksimKorzh wrote:
Sat Oct 17, 2020 8:42 am
Hi guys, thanks to amazing library to probe stockfish NNUE by Daniel Shawul even a noob like me can now get NNUE score for each position, but it turns out that using that score directly leads to a VERY STRANGE behavior of my engine = it makes strange moves from time to time and sacrifices the queen for nothing in every game. I feel a bit confused at this point.

My questions are:
1. Do I need to evaluate material if I want to use pure NNUE eval?
2. Do I need to scale NNUE eval?
3. Do I need to tune material weights? (I heard SF uses around 200 for paen value while my engine does around 100)

Actually what should I do with NNUE score to make my engine play normal chess?

Please help.
FYI I have now added support for the incremental evaluation of NNUE in the library.
This gives a speed bump of 14 to 18%, so maybe worth it for those in need of the highest performance.
The engine has to keep track of dirtyPiece and Accumulator using the same data structure as what the library use.
Make sure accumulator is cache aligned i.e. 64 bytes.
It maybe possible to move some of the code for incremental evaluation from the engine to the library in the future.

For an example implementation of "NNUE incremental" in an engine, look here
https://github.com/dshawul/Scorpio/comm ... 7b7322bf84
If you want "NNUE without incremental evalution", the implementation is much easier
https://github.com/dshawul/Scorpio/comm ... 94cd4b2c5c

I got more that 400 elos out of my implementation with a shared library.

User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 4:49 pm

hgm wrote:
Sat Oct 17, 2020 12:35 pm
Let me get this straight: NNUE is just one term of the Stockfish evaluation, and there are other terms?

Is material the only other term (and is it tapered from opening to end-game)?
Yes, material is the ONLY other term and YES it's tapered from opening to end-game.

User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 4:53 pm

Daniel Shawul wrote:
Sat Oct 17, 2020 3:27 pm
maksimKorzh wrote:
Sat Oct 17, 2020 8:42 am
Hi guys, thanks to amazing library to probe stockfish NNUE by Daniel Shawul even a noob like me can now get NNUE score for each position, but it turns out that using that score directly leads to a VERY STRANGE behavior of my engine = it makes strange moves from time to time and sacrifices the queen for nothing in every game. I feel a bit confused at this point.

My questions are:
1. Do I need to evaluate material if I want to use pure NNUE eval?
2. Do I need to scale NNUE eval?
3. Do I need to tune material weights? (I heard SF uses around 200 for paen value while my engine does around 100)

Actually what should I do with NNUE score to make my engine play normal chess?

Please help.
FYI I have now added support for the incremental evaluation of NNUE in the library.
This gives a speed bump of 14 to 18%, so maybe worth it for those in need of the highest performance.
The engine has to keep track of dirtyPiece and Accumulator using the same data structure as what the library use.
Make sure accumulator is cache aligned i.e. 64 bytes.
It maybe possible to move some of the code for incremental evaluation from the engine to the library in the future.

For an example implementation of "NNUE incremental" in an engine, look here
https://github.com/dshawul/Scorpio/comm ... 7b7322bf84
If you want "NNUE without incremental evalution", the implementation is much easier
https://github.com/dshawul/Scorpio/comm ... 94cd4b2c5c

I got more that 400 elos out of my implementation with a shared library.
Thanks Daniel!
I've annoyed you with couple more issues on github...

Can you please clarify one more question - I still can't figure it out from CFish's source:
Should I use pawn material weight relatively equal to 100 or 200
the reason why I'm asking is here:
[url]view-source:https://hxim.github.io/Stockfish-Evaluation-Guide/[/url]
see line 1153
It says

Code: Select all

var PAWNVALUE = 206;
and this confuses me... may be it's specific to stockgish evaluation guide GUI...

User avatar
hgm
Posts: 25104
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: How to scale stockfish NNUE score?

Post by hgm » Sat Oct 17, 2020 4:56 pm

OK, thanks. I ask because I am interested in trying out this NNUE technique in a new engine, for Janggi. I know absolutely nothing about that game, but I do have a PGN file with 25000 games in it. And perhaps this is enough to train it, and get somewhat reasonable play.

It could also be interesting to try it for Paco Shako.
Get rid of the shit: vote for SHID!

User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 5:15 pm

hgm wrote:
Sat Oct 17, 2020 4:56 pm
OK, thanks. I ask because I am interested in trying out this NNUE technique in a new engine, for Janggi. I know absolutely nothing about that game, but I do have a PGN file with 25000 games in it. And perhaps this is enough to train it, and get somewhat reasonable play.

It could also be interesting to try it for Paco Shako.
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.

User avatar
maksimKorzh
Posts: 348
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 6:41 pm

After a consultation with Daniel Shawul and fixing the bugs I can now say for sure:
1. NNUE scores already incorporate material weights in them
2. Daniel's library returns positive score for whites and negative for black, so no need to alter that
3. Scaling is not needed if pure NNUE eval is used, only for hybrid solutions

May be it helps someone

Daniel Shawul
Posts: 4056
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: How to scale stockfish NNUE score?

Post by Daniel Shawul » Sat Oct 17, 2020 7:21 pm

Some may want to do scaling to get better compatibility with search prunings such as
futility pruning and razoring, etc. where the margins are tuned probably with your old centi-pawn scores.
Other than that, you don't need scaling unless you use classic evaluation in some way.

Post Reply