Page 1 of 2

NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 12:01 am
by maksimKorzh
Hi guys, I'm testing stockfish NNUE embedded into my engine but the endgame performance is incredibly poor.
In the opening and middlegame NNUE obtains huge advantage every single game, but then it comes to the endgame and most games end like this:





Any ideas why?
NNUE can't mate in simple KQK engame at depth 10 ???

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 12:39 am
by maksimKorzh
What I've realized so far is that I need at least 16plies depth search to mate KQK endgame, any lower depth leads to pieces going back and forth without any sense(((

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 1:23 am
by D Sceviour

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 1:26 am
by maksimKorzh
D Sceviour wrote:
Wed Oct 21, 2020 1:23 am
Already tried to point this out.

http://talkchess.com/forum3/viewtopic.p ... 53#p867252
Surprisingly VICE performs slightly better against VICE 1.0 compared to BBC even though being a bit slower

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 4:23 am
by RubiChess
Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 9:42 am
by Raphexon
RubiChess wrote:
Wed Oct 21, 2020 4:23 am
Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas
Mostly, but SF for example has some very simple heuristics to aid in finding the mate in such situations. (Like KRvK and KQvK endgames)
And if you replace the whole eval with NNUE you also throw away those heuristics.

But yea, the search should be able to find it.

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 10:02 am
by towforce
RubiChess wrote:
Wed Oct 21, 2020 4:23 am
Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

In an ideal world, a position which is closer to checkmate would get a higher valuation.

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 10:25 am
by mvanthoor
towforce wrote:
Wed Oct 21, 2020 10:02 am
In an ideal world, a position which is closer to checkmate would get a higher valuation.
However, a chess engine doesn't know any mating principles (if you don't program them in somehow). It doesn't know what 'closer to mate' is; to be able to know that it is mate, it has to find it from the starting position. A KR-k mate is a mate in 17 (or thereabouts), which is quite hard to find.

In my chess engine, the PSQT's (it only still has one set) prevent it from finding mate because it wants to keep the king on its own side of the board, and the rook on the d or e-file. It isn't able to search 17+ plies yet, so it isn't able to see the mate, and it also won't drive the king to the side of the board. Neural networks, due to the low(er) search depth, are notorious for poor endgame performance. Even a/b-engines are often not very good at it, because the endgame mostly doesn't contain a lot of tactics; humans play it on knowledge and principles.

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 11:11 am
by towforce
mvanthoor wrote:
Wed Oct 21, 2020 10:25 am
towforce wrote:
Wed Oct 21, 2020 10:02 am
In an ideal world, a position which is closer to checkmate would get a higher valuation.
However, a chess engine doesn't know any mating principles (if you don't program them in somehow). It doesn't know what 'closer to mate' is; to be able to know that it is mate, it has to find it from the starting position. A KR-k mate is a mate in 17 (or thereabouts), which is quite hard to find.

In my chess engine, the PSQT's (it only still has one set) prevent it from finding mate because it wants to keep the king on its own side of the board, and the rook on the d or e-file. It isn't able to search 17+ plies yet, so it isn't able to see the mate, and it also won't drive the king to the side of the board. Neural networks, due to the low(er) search depth, are notorious for poor endgame performance. Even a/b-engines are often not very good at it, because the endgame mostly doesn't contain a lot of tactics; humans play it on knowledge and principles.

Trying to avoid a hijack to a repeat of the "can chess be solved" thread, but if humans can play endgames "reasonably" then you'd expect that NNs could be trained to as well. I understand that the state-of-the-art right now is that solving endgames (as opposed to looking them up) still requires deep searching.

Re: NNUE incredibly poor endgame performance

Posted: Wed Oct 21, 2020 11:58 am
by mar
RubiChess wrote:
Wed Oct 21, 2020 4:23 am
Simple mating is a search task. I wouldn't blame the evaluation not finding a mate.

Andreas
exactly. if the engine can't find a mate in 3 in KQK endgame in 10 plies, it should be fixed.
no matter what the eval returns, whether it's 0 or random garbage.