Page 2 of 2

Re: How to scale stockfish NNUE score?

Posted: Sat Oct 17, 2020 7:23 pm
by hgm
maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.

Re: How to scale stockfish NNUE score?

Posted: Sat Oct 17, 2020 9:54 pm
by maksimKorzh
hgm wrote:
Sat Oct 17, 2020 7:23 pm
maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.
Thanks for explanations. Very interesting! It would be even more interesting if you were making microMax like tutorials - I've learned so much from that tutorial! Your answers here are for good programmers (not for me) but your tutorials are great!

Re: How to scale stockfish NNUE score?

Posted: Tue Oct 20, 2020 7:20 pm
by maksimKorzh
I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???

Re: How to scale stockfish NNUE score?

Posted: Wed Oct 21, 2020 9:45 am
by Raphexon
hgm wrote:
Sat Oct 17, 2020 7:23 pm
maksimKorzh wrote:
Sat Oct 17, 2020 5:15 pm
I am always getting excited when you're trying to implement engines for games other but chess. I'm just wondering what motivates you?
I mean usually people developing one engine for years and it's difficult to master chess programming at a decent level...
But you're trying many other games! This is incredible, so what motivates you? Thanks in advance.
I am driven mainly by curiosity, I like doing creative work, and it is gratifying to create things that others are eager to use. That sort of disqualifies orthodox Chess, for which there are already thousands of engines. Except for a handful of testers no one is interested in having yet another one. So I try to focus on games where nothing, or nothing decent exists yet. Or where it is truly unknown what is a good way to do it.

I find Paco Shako a very intriguing game in this respect; no one really knows how to play that. I am very curious what would be a good strategy for it. This NNUE method seems ideally suited for it, and material does not seem to be an important term in that game (it is impossible to lose any...), and King Safety is everything. And as I happen to have a lot of Janggi games handy, this seems a good opportunity to get some hands-on experience with this interesting NNUE technique. For Janggi there seems to hardly exist anything, and millions of potential users.
Yea, this was my main reason for initial excitement for NNUE.
It really eases engine development for alternative board games (or chess variants) since eval is no longer difficult.

Re: How to scale stockfish NNUE score?

Posted: Thu Oct 22, 2020 3:14 pm
by Madeleine Birchfield
maksimKorzh wrote:
Tue Oct 20, 2020 7:20 pm
I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???
Stockfish NNUE might be trained with games that are adjudicated at the 7 men tablebase scoring instead of at checkmate, meaning that it never saw any positions with seven or less pieces and therefore doesn't know how to deal with them.

Re: How to scale stockfish NNUE score?

Posted: Thu Oct 22, 2020 3:20 pm
by Madeleine Birchfield
hgm wrote:
Sat Oct 17, 2020 4:56 pm
OK, thanks. I ask because I am interested in trying out this NNUE technique in a new engine, for Janggi. I know absolutely nothing about that game, but I do have a PGN file with 25000 games in it. And perhaps this is enough to train it, and get somewhat reasonable play.

It could also be interesting to try it for Paco Shako.
You could also use reinforcement learning

Re: How to scale stockfish NNUE score?

Posted: Thu Oct 22, 2020 9:49 pm
by JohnWoe
Madeleine Birchfield wrote:
Thu Oct 22, 2020 3:14 pm
maksimKorzh wrote:
Tue Oct 20, 2020 7:20 pm
I'm facing some very weird behavior of Stockfish NNUE in my engine:
IT'S VERY POOR IN PLAYING ENDGAMES

e.g. my old noob's handcrafted eval mated with king queen and knight vs king EASILY while NNUE version manages to draw with 50 move rule.
I didn't touch search.

How on earth is tha possible???
Stockfish NNUE might be trained with games that are adjudicated at the 7 men tablebase scoring instead of at checkmate, meaning that it never saw any positions with seven or less pieces and therefore doesn't know how to deal with them.
On my engine Mayhem it also apparent it wasn't trained on Chess960 positions. Also not endgames. I have a small "classical" evaluation function to finish the game. Otherwise it provides very accurate MG evals.