kpk bitbase, worthless?

[brtzsnr]

With the impending start of CENT, I finally decided to bite the bullet and implement KPK bitbases. I have an implementation that works correctly (verified with pfkpk from Marcel van Kervinck).

My scoring function looks like this now

Code: Select all

func (eng *Engine) Score() int32 {
	outcome := endgame(eng.Position)
	if outcome == draw {
		return 0
	}

	score := Evaluate(eng.Position)
	score *= eng.Position.Us().Multiplier()

	if outcome == win {
		score += 100
	} else if outcome == loss {
		score -= 100
	}

	return score
}

In plain words: if the position is a draw return 0. If the position is winning or losing adjust the score accordingly.

Unfortunately, it seems that having kpk doesn't improve playing strength at all:

STC
4930 @ 40/15+0.05
1619 - 1622 - 1689
ELO -0.21±7.86

Also, testing 500 5-men positions (from Kai) the improvement in number of nodes searched is slightly larger (bitbase: 70169326, master: 68455885).

Am I missing anything obvious in they way I use the kpk bitbase? Or is the bitbase worthless?


[1] https://bitbucket.org/brtzsnr/zurichess ... ew-default

[cdani]

When I implemented kpk bitbase I was not doing tests seriously, so I cannot say if it was a win. But I can say thay I return 0 for draw, or base_value_win (6000) if it's won, where the engine adds to those 6000 the standard evaluation. I never had a problem or bad played kpk endgame since that I know.

For example here it does Nf3xg5:
[d]8/8/6k1/4P1p1/8/4KN2/8/8 w - - 0 1

[brtzsnr]

Ke4 (found without bitbase) and Nd4 (found with bitbase) are both good moves verified with Stockfish.

Edit: I add a small value because I haven't implemented KQK. With a large value the engine refuses to promote because it doesn't know that KQK is winning (mostly).

[cdani]

Ke4 (found without bitbase) and Nd4 (found with bitbase) are both good moves verified with Stockfish.

Edit: I add a small value because I haven't implemented KQK. With a large value the engine refuses to promote because it doesn't know that KQK is winning (mostly).

In understand. Ng5 is the normal answer when one adds 6000 to the resulting endgame.
Sure is important to have implemented well all those basic draw/win endgames like KQK, so you can do well the code of the other endgames that can end in those more simple.

[Dirt]

Or is the bitbase worthless?

In many positions, but not all, it is worthless. If every move is a win it tells you nothing about how to proceed. Once your root position is in the bitbase you need some kind of DTM or something to continue. Syzygy provides this. Rolling your own is difficult.

[cdani]

Rolling your own is difficult.

Well, in the case of kpk is easy. Just the passed pawn evaluation is enough.

[PK]

Rodent uses poor man's replacement of kpk bitbase - it detects some basic draws when the winning side cannot make progress. I'm not sure if it gains Elo, but I keep that code, so that weaker Rodent levels with limited search speed don't appear too stupid.

[tpetzke]

I think overall up to 5-men EGTB have not so much influence on ELO (with KRPKR as a possible exception). It just leads to a cleaner play by the engine and avoids situations where some user posts a position in a forum and states your engine is stupid and gave away a clear win or so.

iCE calculates the KPK tablebase when the engine starts in memory. But the main reason for doing it was not ELO. I just liked playing with the theory of TBs.

[Evert]

You gain an advantage from KPK in cases where it helps the engine to avoid trading into a drawn position from a won position, or "losing the draw".

Otherwise it doesn't help.

If the majority of games are not decided by KPK (which is likely) the effect it has is always going to be small to begin with. Add to that that it will only help in positions where the engine would not win when it could or preserve the draw, which are a subset of KPK positions, and the effect becomes smaller still. To test if it makes you engine stronger you should first play late end-game positions that convert to KPK (both won and drawn positions) against an opponent who will play them correctly. Then you can compare the result against what you expect (ie, your engine should win 100% of won positions against the other side and draw 100% of the drawn positions). Getting a statistically significant number of independent positions (let alone games) is going to be hard. If you pass that, you know it plays KPK correctly. The only thing that can make it not an Elo gain then is if it slows you down too much, which seems unlikely but can also be tested.

Bottom line: since the expected gain from this is tiny, testing for Elo improvement is pointless here.

[hgm]

It will also depend a lot on how you evaluate KPK positions in absence of the bitbase. If you simply score them as 150 plus some bonus for how far they are advanced, not even looking if it is an unstoppable passer, there would be more to gain then when you do recognize unstoppable passers.