NNUE incredibly poor endgame performance

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
maksimKorzh
Posts: 551
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:
Contact maksimKorzh

NNUE incredibly poor endgame performance

Post by maksimKorzh » Wed Oct 21, 2020 12:01 am

Hi guys, I'm testing stockfish NNUE embedded into my engine but the endgame performance is incredibly poor.
In the opening and middlegame NNUE obtains huge advantage every single game, but then it comes to the endgame and most games end like this:





Any ideas why?
NNUE can't mate in simple KQK engame at depth 10 ???
JavaScript chess engine with UCI support, own GUI and public API:
https://github.com/maksimKorzh/wukongJS

Chess programming YouTube channel:
https://www.youtube.com/channel/UCB9-pr ... KKqDgXhsMQ
Top
User avatar
maksimKorzh
Posts: 551
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:
Contact maksimKorzh

Re: NNUE incredibly poor endgame performance

Post by maksimKorzh » Wed Oct 21, 2020 12:39 am

What I've realized so far is that I need at least 16plies depth search to mate KQK endgame, any lower depth leads to pieces going back and forth without any sense(((
JavaScript chess engine with UCI support, own GUI and public API:
https://github.com/maksimKorzh/wukongJS

Chess programming YouTube channel:
https://www.youtube.com/channel/UCB9-pr ... KKqDgXhsMQ
Top
D Sceviour
Posts: 568
Joined: Mon Jul 20, 2015 3:06 pm
Contact:
Contact D Sceviour

Re: NNUE incredibly poor endgame performance

Post by D Sceviour » Wed Oct 21, 2020 1:23 am

Already tried to point this out.

http://talkchess.com/forum3/viewtopic.p ... 53#p867252
Top
User avatar
maksimKorzh
Posts: 551
Joined: Sat Sep 08, 2018 3:37 pm
Location: Ukraine
Full name: Maksim Korzh
Contact:
Contact maksimKorzh

Re: NNUE incredibly poor endgame performance

Post by maksimKorzh » Wed Oct 21, 2020 1:26 am

D Sceviour wrote: ↑
Wed Oct 21, 2020 1:23 am
 Already tried to point this out.

http://talkchess.com/forum3/viewtopic.p ... 53#p867252
Surprisingly VICE performs slightly better against VICE 1.0 compared to BBC even though being a bit slower
JavaScript chess engine with UCI support, own GUI and public API:
https://github.com/maksimKorzh/wukongJS

Chess programming YouTube channel:
https://www.youtube.com/channel/UCB9-pr ... KKqDgXhsMQ
Top
RubiChess
Posts: 259
Joined: Fri Mar 30, 2018 5:20 am
Full name: Andreas Matthies

Re: NNUE incredibly poor endgame performance

Post by RubiChess » Wed Oct 21, 2020 4:23 am

Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas
Top
Raphexon
Posts: 351
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: NNUE incredibly poor endgame performance

Post by Raphexon » Wed Oct 21, 2020 9:42 am

RubiChess wrote: ↑
Wed Oct 21, 2020 4:23 am
 Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas
Mostly, but SF for example has some very simple heuristics to aid in finding the mate in such situations. (Like KRvK and KQvK endgames)
And if you replace the whole eval with NNUE you also throw away those heuristics.

But yea, the search should be able to find it.
Top
User avatar
towforce
Posts: 10694
Joined: Wed Mar 08, 2006 11:57 pm
Location: Birmingham UK

Re: NNUE incredibly poor endgame performance

Post by towforce » Wed Oct 21, 2020 10:02 am

RubiChess wrote: ↑
Wed Oct 21, 2020 4:23 am
 Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

In an ideal world, a position which is closer to checkmate would get a higher valuation.
Writing is the antidote to confusion
Top
User avatar
mvanthoor
Posts: 598
Joined: Wed Jul 03, 2019 2:42 pm
Full name: Marcel Vanthoor

Re: NNUE incredibly poor endgame performance

Post by mvanthoor » Wed Oct 21, 2020 10:25 am

towforce wrote: ↑
Wed Oct 21, 2020 10:02 am
 In an ideal world, a position which is closer to checkmate would get a higher valuation.
However, a chess engine doesn't know any mating principles (if you don't program them in somehow). It doesn't know what 'closer to mate' is; to be able to know that it is mate, it has to find it from the starting position. A KR-k mate is a mate in 17 (or thereabouts), which is quite hard to find.

In my chess engine, the PSQT's (it only still has one set) prevent it from finding mate because it wants to keep the king on its own side of the board, and the rook on the d or e-file. It isn't able to search 17+ plies yet, so it isn't able to see the mate, and it also won't drive the king to the side of the board. Neural networks, due to the low(er) search depth, are notorious for poor endgame performance. Even a/b-engines are often not very good at it, because the endgame mostly doesn't contain a lot of tactics; humans play it on knowledge and principles.
Top
User avatar
towforce
Posts: 10694
Joined: Wed Mar 08, 2006 11:57 pm
Location: Birmingham UK

Re: NNUE incredibly poor endgame performance

Post by towforce » Wed Oct 21, 2020 11:11 am

mvanthoor wrote: ↑
Wed Oct 21, 2020 10:25 am
towforce wrote: ↑
Wed Oct 21, 2020 10:02 am
 In an ideal world, a position which is closer to checkmate would get a higher valuation.
However, a chess engine doesn't know any mating principles (if you don't program them in somehow). It doesn't know what 'closer to mate' is; to be able to know that it is mate, it has to find it from the starting position. A KR-k mate is a mate in 17 (or thereabouts), which is quite hard to find.

In my chess engine, the PSQT's (it only still has one set) prevent it from finding mate because it wants to keep the king on its own side of the board, and the rook on the d or e-file. It isn't able to search 17+ plies yet, so it isn't able to see the mate, and it also won't drive the king to the side of the board. Neural networks, due to the low(er) search depth, are notorious for poor endgame performance. Even a/b-engines are often not very good at it, because the endgame mostly doesn't contain a lot of tactics; humans play it on knowledge and principles.

Trying to avoid a hijack to a repeat of the "can chess be solved" thread, but if humans can play endgames "reasonably" then you'd expect that NNs could be trained to as well. I understand that the state-of-the-art right now is that solving endgames (as opposed to looking them up) still requires deep searching.
Writing is the antidote to confusion
Top
mar
Posts: 2262
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: NNUE incredibly poor endgame performance

Post by mar » Wed Oct 21, 2020 11:58 am

RubiChess wrote: ↑
Wed Oct 21, 2020 4:23 am
 Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas
exactly. if the engine can't find a mate in 3 in KQK endgame in 10 plies, it should be fixed.
no matter what the eval returns, whether it's 0 or random garbage.
Martin Sedlak
Top
Post Reply

Return to “Computer Chess Club: Programming and Technical Discussions”