Obteining conclusion of lossed games

[Kempelen]

How do you conclude when you engine is weak in the search versus the eval function?

I examined a few lossed games played by Rodin and I have sometimes no clue if I need to improve the search or the eval. It is easy to confuse a position been bad because you did not tune the eval correctly when in fact the problem is a tactical flaw.

[Gian-Carlo Pascutto]

This is a very good question...

[Edmund]

How do you conclude when you engine is weak in the search versus the eval function?

I examined a few lossed games played by Rodin and I have sometimes no clue if I need to improve the search or the eval. It is easy to confuse a position been bad because you did not tune the eval correctly when in fact the problem is a tactical flaw.

You could try using a multi-pv search. Then you will see how much under-evaluated the best move actually is.

[Kempelen]

You could try using a multi-pv search. Then you will see how much under-evaluated the best move actually is.

What exactly is MultiPV and how can it be implemented? (a fast search in "Chess programming Wiki" and the forum do not show many results)

[Edmund]

MultiPV means that the engine not only returns the bestmove, but all n best moves in a certain position + its PV ans score.

it is implemented by at the root node not passing alpha, beta but alpha_of_the_nth_best_move, beta .. thus you receive exact scores also for inferior moves. secondly you have to somehow collect the pvs of these moves too

with UCI protocol it is implemented like this:

Code: Select all

22.422-->1:uci
22.861<--1&#58;id name Glass 1.3
22.923<--1&#58;id author Pawel Koziol and Edmund Moshammer
&#91;...&#93;
23.033<--1&#58;option name MultiPV type spin default 1 min 1 max 255
&#91;...&#93;
23.251<--1&#58;uciok
&#91;...&#93;
32.829-->1&#58;setoption name MultiPV value 5
39.485-->1&#58;isready
39.532<--1&#58;readyok
39.547-->1&#58;ucinewgame
39.564-->1&#58;isready
39.579<--1&#58;readyok
39.594-->1&#58;position startpos
39.625-->1&#58;go infinite
40.283<--1&#58;info multipv 1 depth 1 score cp 46 time 609 nodes 2 nps 3 pv e2e4
40.611<--1&#58;info multipv 2 depth 1 score cp 46 time 625 nodes 3 nps 4 pv d2d4
40.657<--1&#58;info multipv 3 depth 1 score cp 30 time 625 nodes 4 nps 6 pv g1f3
40.689<--1&#58;info multipv 4 depth 1 score cp 30 time 625 nodes 5 nps 8 pv b1c3
40.766<--1&#58;info multipv 5 depth 1 score cp 28 time 625 nodes 6 nps 9 pv e2e3
40.782<--1&#58;info multipv 1 depth 2 score cp 12 time 640 nodes 43 nps 67 pv e2e4 e7e5
40.829<--1&#58;info multipv 2 depth 2 score cp 12 time 640 nodes 65 nps 101 pv d2d4 d7d5
40.861<--1&#58;info multipv 3 depth 2 score cp -4 time 640 nodes 87 nps 135 pv g1f3 d7d5
40.922<--1&#58;info multipv 4 depth 2 score cp -6 time 640 nodes 108 nps 168 pv b1c3 d7d5
40.986<--1&#58;info multipv 5 depth 2 score cp -7 time 640 nodes 129 nps 201 pv e2e3 d7d5
41.001<--1&#58;info multipv 1 depth 3 score cp 21 time 656 nodes 311 nps 474 pv e2e4 d7d5 f1b5
41.079<--1&#58;info multipv 1 depth 3 score cp 28 time 656 nodes 380 nps 579 pv d2d4 d7d5 g1f3
41.172<--1&#58;info multipv 2 depth 3 score cp 21 time 656 nodes 380 nps 579 pv e2e4 d7d5 f1b5
41.204<--1&#58;info multipv 2 depth 3 score cp 28 time 656 nodes 443 nps 675 pv g1f3 d7d5 d2d4
41.267<--1&#58;info multipv 3 depth 3 score cp 21 time 656 nodes 443 nps 675 pv e2e4 d7d5 f1b5
41.360<--1&#58;info multipv 4 depth 3 score cp 17 time 656 nodes 510 nps 777 pv b1c3 d7d5 d2d4
41.470<--1&#58;info multipv 5 depth 3 score cp 17 time 656 nodes 586 nps 893 pv e2e3 d7d5 d2d4
41.517<--1&#58;info multipv 1 depth 4 score cp 10 time 656 nodes 1481 nps 2257 pv d2d4 d7d5 g1f3 g8f6
41.610<--1&#58;info multipv 2 depth 4 score cp 10 time 656 nodes 1939 nps 2955 pv g1f3 d7d5 d2d4 g8f6
41.658<--1&#58;info multipv 3 depth 4 score cp 7 time 656 nodes 2795 nps 4260 pv e2e4 d7d5 b1c3 d5e4
41.766<--1&#58;info multipv 4 depth 4 score cp 7 time 656 nodes 2997 nps 4568 pv b1c3 d7d5 e2e4 d5e4
41.845<--1&#58;info multipv 5 depth 4 score cp -1 time 671 nodes 3609 nps 5378 pv e2e3 d7d5 d2d4 g8f6
41.923<--1&#58;info multipv 1 depth 5 score cp 23 time 687 nodes 6214 nps 9045 pv d2d4 d7d5 g1f3 g8f6 c1f4
41.986<--1&#58;info multipv 2 depth 5 score cp 23 time 687 nodes 7588 nps 11045 pv g1f3 d7d5 d2d4 g8f6 c1f4
42.141<--1&#58;info multipv 3 depth 5 score cp 21 time 718 nodes 10391 nps 14472 pv e2e4 d7d5 e4d5 d8d5 d2d4
42.189<--1&#58;info multipv 4 depth 5 score cp 15 time 718 nodes 11191 nps 15586 pv b1c3 d7d5 d2d4 g8f6 g1f3
42.282<--1&#58;info multipv 5 depth 5 score cp 12 time 734 nodes 13185 nps 17963 pv e2e3 d7d5 d2d4 g8f6 g1f3
42.329<--1&#58;info multipv 1 depth 6 score cp 10 time 781 nodes 28567 nps 36577 pv d2d4 d7d5 g1f3 g8f6 c1f4 c8f5
42.391<--1&#58;info multipv 2 depth 6 score cp 10 time 796 nodes 35188 nps 44206 pv g1f3 d7d5 d2d4 g8f6
42.532<--1&#58;info multipv 3 depth 6 score cp 5 time 843 nodes 46370 nps 55005 pv e2e4 d7d5 e4e5 c8f5 d2d4 e7e6
42.657<--1&#58;info multipv 4 depth 6 score cp 4 time 843 nodes 50482 nps 59883 pv b1c3 d7d5 e2e4 g8f6 e4d5 f6d5
42.735<--1&#58;info multipv 5 depth 6 score cp 0 time 890 nodes 62166 nps 69849 pv e2e3 d7d5 d2d4 g8f6 g1f3 c8f5
43.033<--1&#58;info multipv 1 depth 7 score cp 19 time 1218 nodes 103000 nps 84564 pv d2d4 d7d5 g1f3 g8f6 c1f4 c8f5 e2e3
43.313<--1&#58;info multipv 1 depth 7 score cp 28 time 1421 nodes 138698 nps 97605 pv g1f3 b8c6 d2d4 d7d5 c1f4 g8f6 e2e3
43.500<--1&#58;info multipv 2 depth 7 score cp 19 time 1421 nodes 138698 nps 97605 pv d2d4 d7d5 g1f3 g8f6 c1f4 c8f5 e2e3
43.563<--1&#58;info multipv 3 depth 7 score cp 16 time 1593 nodes 167580 nps 105197 pv e2e4 d7d5 e4e5 c8f5 d2d4 e7e6 g1f3
43.766<--1&#58;info multipv 3 depth 7 score cp 17 time 1671 nodes 178005 nps 106526 pv b1c3 d7d5 d2d4 g8f6 g1f3 c8f5 c1f4
43.986<--1&#58;info multipv 4 depth 7 score cp 16 time 1671 nodes 178005 nps 106526 pv e2e4 d7d5 e4e5 c8f5 d2d4 e7e6 g1f3
44.032<--1&#58;info multipv 3 depth 7 score cp 19 time 1890 nodes 223094 nps 118039 pv e2e3 e7e5 d2d4 e5e4 d1g4 d7d5 f1b5
44.282<--1&#58;info multipv 4 depth 7 score cp 17 time 1890 nodes 223094 nps 118039 pv b1c3 d7d5 d2d4 g8f6 g1f3 c8f5 c1f4
44.407<--1&#58;info multipv 5 depth 7 score cp 16 time 1890 nodes 223094 nps 118039 pv e2e4 d7d5 e4e5 c8f5 d2d4 e7e6 g1f3
44.875<--1&#58;info multipv 1 depth 8 score cp 8 time 2718 nodes 383250 nps 141004 pv g1f3 g8f6 e2e3 e7e6 f1e2 f8d6 e1g1 e8g8
45.016-->1&#58;stop
45.704<--1&#58;bestmove g1f3 ponder g8f6

[Antonio Torrecillas]

How do you conclude when you engine is weak in the search versus the eval function?

The right answer must be debugging. The question become, how to get positions badly evaluated/searched that are easy to debug ?
Let me recall the know post about evaluation from Tord.
http://www.talkchess.com/forum/viewtopi ... 33&t=15504
Now if you think “continuity” and “Good worst case“ replacing evaluation by search, all apply?.
That the base for a method I use:
Traverse a bunch of position and do a depth 2 search on each.
You can instrument your iterative deepening function and calculate:

Code: Select all

double dif = abs(Value_at_depth_1 - Value_at_depth_2);

and separate the big diferences, (exclude mates values), I hope that, in this position there is a big Tactic, a bug....

Code: Select all

if(SaveBadEval && Depth == 2 && dif > MargenError
	&& BestMove.desglose.jaque == 0
	&& BestMove.desglose.captura == ninguna
	&& BestMove.desglose.coronar == ninguna
	)
{
	FILE *fe = fopen("badeval.epd","a+");
	if(fe)
	{
		fprintf(fe,"%s bm %s;\n",Board.SaveFEN(),BestMove);
		fclose(fe);
	}
	FILE *fl = fopen("badeval.log","a+");
	if(fl)
	{
		fprintf(fl,"%s bm %s;\n",Board.SaveFEN(),BestMove);
		fprintf(fl,"PV %s Eval %d dif %4.0lf;\n",SavePV,value,dif);
		Board.Draw(fl);
		fclose(fl);
	}
}

Look at the saved position, if something is obviously wrong, start to debug.

Buena Suerte.