Discocheck 5.01: Bishop related endgame problems

[Uri Blass]

It doesn't value the King!

(OK, cheap shot... But seriously, this is very elementary, and I think it really hurts Elo if an engine does not recognize positions without mating potential for the leading side. Just recognizing KBK and KNK as draws isn't enough. End-games like KNPKN where the Pawn can be trivially blocked by the threat of an NxP sacrifice are quite common, and end-games of minor + Pawn vs minor + Pawn in which you cannot avoid trading into minor vs Pawn are not rare either. A leading side without mating potential is as good as dead.)

A single knight may be enough for mating potential when the opponent has a pawn.

[d]8/8/8/4N3/8/p7/k1K5/8 w - - 0 1

1.Nd3 Ka1 2.Nc1 a2 Nb3 mate.

KN vs K is a draw
KN vs KP give some chances for the knight to win

Another example:
Here tablebases say that the only way to win is Na2 and after Kxa2 you get the first diagram
[d]8/8/8/4N3/8/p1N5/2K5/k7 w - - 0 5

[hgm]

This is not really a problem, as the multipliers are not used in the context of recognizers, but just modify normal evaluation. So they won't block further searching of the branch. If you hit an exception like this, at first it would reduce the score because it expects it to be drawish. But on deeper search it would see the tactical win. This kind of behavior is quite common in searches, where you often have to sacrifice something to later gain back even more. Of course it would be nicer if the evaluation gave you immediately the correct value, without any search. But we know that this in general is not possible.

The purpose of evaluation is more to include those features that no reasonable amount of search would ever find. E.g. without the knowledge in eval, the engine would only realize KBK is a draw when it reaches 100 ply, and runs into the 50-move limit.

If you don't have the depth to see the mate in a branch that converted to the KNKP position you gave, the engine would avoid it. Which is probably sensible behavior, as most of the positions with this material signature would be draw. So you should not go there without first making sure it is a win after all, unless all alternatives are even worse than the discounted score.

[lucasart]

OK, I finally decided to partially fix this problem. But only by adding general knowledge, nothing specific.

DiscoCheck now has the following endgame knowledge:

Code: Select all

if (stronger side has no pawns) {
	if (stronger side has mating material) {
		if (weaker side has more than just a king)
			eval_factor = 0.5
	} else
		eval_factor = 0.25	// added this rule
}

Now, I'm running a test of this entire endgame scaling factor logic: not just the new rule but the whole lot. If it fails to improve elo, as I strongly suspect, I will remove the whole lot, and DiscoCheck will have absolutely no endgame knowledge left

There are so many myths in computer chess, and so many people who prefer to believe those myths rather than empirical evidence. I do not believe in anything "a priori", however appealing and intuitive it may be: if it does not improve elo is has no place in DiscoCheck. It's that simple.

[hgm]

You could also remove e.p. capture. That also won't bring you measurable Elo. And of course under-promotion. But maybe you got rid of that already.

[Joerg Oster]

And playing games against other, more or less stupid engines, delivers empirical evidence?

No offense meant, Lucas, but I really think only looking at Elo-gain in engine-development, is a major mistake nowadays.

[lucasart]

OK, I finally decided to partially fix this problem. But only by adding general knowledge, nothing specific.

DiscoCheck now has the following endgame knowledge:

Code: Select all

if (stronger side has no pawns) {
	if (stronger side has mating material) {
		if (weaker side has more than just a king)
			eval_factor = 0.5
	} else
		eval_factor = 0.25	// added this rule
}

Now, I'm running a test of this entire endgame scaling factor logic: not just the new rule but the whole lot. If it fails to improve elo, as I strongly suspect, I will remove the whole lot, and DiscoCheck will have absolutely no endgame knowledge left

There are so many myths in computer chess, and so many people who prefer to believe those myths rather than empirical evidence. I do not believe in anything "a priori", however appealing and intuitive it may be: if it does not improve elo is has no place in DiscoCheck. It's that simple.

Actually, it turns out that the above code snippet has some measurable elo value. Nothing big, of course, but measurable with my testing resolution (SPRT elo0=0 elo1=4 alpha=beta=5%). So the patch is commited.

[lucasart]

You could also remove e.p. capture. That also won't bring you measurable Elo. And of course under-promotion. But maybe you got rid of that already.

I know you're joking, but I will answer seriously, because this is a thought provoking comment.

1/ En passant:

"removing en passant" would not lead to any code simplification. Quite the contrary. What happens is that you have to parse and play the moves from the UCI position commnad. So if your opponent plays en passant moves, you have to keep the code to play ep moves. Also, you need to generate them, because you need to search ep moves at the root node, or else the engine crashes when ep is the only legal move. That also means you have to keep the code to undo en passant moves. Conclusion: zero code removed, all you do is add code that arbitrarily excludes en passant at all non root nodes.

Being a complexity adding patch, rather than a simplification patch, it has to demonstrate an elo gain. IOW empirical evidence has to show a measurable gain from pruning ep captures at all non root nodes. I have not tested this, but I doubt the result would be positive.

2/ Promotion:

You seem to be misguided by the intuition that pruning under promotion gains elo (or at least doesn't lose any), because it reduces the branching factor while incurring no bad moves. This would indeed be logical. But the reality it different! I have tested this several times, and the conclusion is always the same:
(i) pruning under promotions in the search() is measurably regressive.
(ii)* pruning under promotions in the qsearch() is, as expected, a measurabel (albeit small) gain.

(i) is indeed a very surprising result, and I was shocked, so I redid the experiment only to find the same conclusion. I do not claim to understand why. But facts are what they are, whether we understand them or not does not rule them in or out of existence.

[hgm]

So if your opponent plays en passant moves, you have to keep the code to play ep moves.

No, you can just exit and forfeit. Or resign (bestmove 0000).

[syzygy]

And playing games against other, more or less stupid engines, delivers empirical evidence?

No offense meant, Lucas, but I really think only looking at Elo-gain in engine-development, is a major mistake nowadays.

I agree. Apparently some are removing basic endgame knowledge that cannot possibly hurt, only because they cannot measure the 0.1 Elo decrease per micropatch.

Common sense, when available, should always prevail. Certain endgames simply don't happen often enough to measurably influence a test of thousands of games. But common sense should tell that there is no point in removing a few lines of code that improve such an endgame and are not even executed when the endgame is not seen.