How to scale stockfish NNUE score?

Discussion of chess software programming and technical issues.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
hgm
Posts: 25049
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: How to scale stockfish NNUE score?

Post by hgm » Sat Oct 17, 2020 7:23 pm

maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.
Get rid of the shit: vote for SHID!

User avatar
maksimKorzh
Posts: 334
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Sat Oct 17, 2020 9:54 pm

hgm wrote:
Sat Oct 17, 2020 7:23 pm
maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.
Thanks for explanations. Very interesting! It would be even more interesting if you were making microMax like tutorials - I've learned so much from that tutorial! Your answers here are for good programmers (not for me) but your tutorials are great!

User avatar
maksimKorzh
Posts: 334
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:

Re: How to scale stockfish NNUE score?

Post by maksimKorzh » Tue Oct 20, 2020 7:20 pm

I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???

Raphexon
Posts: 327
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: How to scale stockfish NNUE score?

Post by Raphexon » Wed Oct 21, 2020 9:45 am

hgm wrote:
Sat Oct 17, 2020 7:23 pm
maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.
Yea, this was my main reason for initial excitement for NNUE.
It really eases engine development for alternative board games (or chess variants) since eval is no longer difficult.

Madeleine Birchfield
Posts: 76
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: How to scale stockfish NNUE score?

Post by Madeleine Birchfield » Thu Oct 22, 2020 3:14 pm

maksimKorzh wrote:
Tue Oct 20, 2020 7:20 pm
I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???
Stockfish NNUE might be trained with games that are adjudicated at the 7 men tablebase scoring instead of at checkmate, meaning that it never saw any positions with seven or less pieces and therefore doesn't know how to deal with them.

Madeleine Birchfield
Posts: 76
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: How to scale stockfish NNUE score?

Post by Madeleine Birchfield » Thu Oct 22, 2020 3:20 pm

hgm wrote:
Sat Oct 17, 2020 4:56 pm
OK, thanks. I ask because I am interested in trying out this NNUE technique in a new engine, for Janggi. I know absolutely nothing about that game, but I do have a PGN file with 25000 games in it. And perhaps this is enough to train it, and get somewhat reasonable play.

It could also be interesting to try it for Paco Shako.
You could also use reinforcement learning

JohnWoe
Posts: 228
Joined: Sat Mar 02, 2013 10:31 pm

Re: How to scale stockfish NNUE score?

Post by JohnWoe » Thu Oct 22, 2020 9:49 pm

Madeleine Birchfield wrote:
Thu Oct 22, 2020 3:14 pm
maksimKorzh wrote:
Tue Oct 20, 2020 7:20 pm
I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???
Stockfish NNUE might be trained with games that are adjudicated at the 7 men tablebase scoring instead of at checkmate, meaning that it never saw any positions with seven or less pieces and therefore doesn't know how to deal with them.
On my engine Mayhem it also apparent it wasn't trained on Chess960 positions. Also not endgames. I have a small "classical" evaluation function to finish the game. Otherwise it provides very accurate MG evals.

Post Reply