Tuning eval to minimize nodes searched

[FlavusSnow]

Can tuning the evaluation function cause more cut-offs and thus reduce nodes searched for a certain depth?

I've often assumed that to properly tune eval, you'd need fixed depth matches, but this morning I thought that maybe a properly tuned evaluation function merely facilitates a faster search by reducing the nodes searched.

[Desperado]

Can tuning the evaluation function cause more cut-offs and thus reduce nodes searched for a certain depth?

I've often assumed that to properly tune eval, you'd need fixed depth matches, but this morning I thought that maybe a properly tuned evaluation function merely facilitates a faster search by reducing the nodes searched.

It can, but of course the opposite can also be.
- Using fix depth matches may give you an idea if your feature is worth something.
- dont forget that evaluation can be sensitive to depth, so you may
get different effects on different depth-matches.
- depth wont give you an answer, if the feature gain is worth the time loss. (the costs to compute the feature).
- finally you want to know the average of the above points, which imho
needs a time-based validation test anyway.

So, there are features reducing the tree size and finally raise the quality of chess knowledge.
But others will only do one of the two points, in bad case none of it.
In any case you have to know how it effects on average the playing strength under real competition conditions.

Michael

[hgm]

Indeed, better tuned eval is reported to give an enormous reduction of the search tree, with modern search methods.

[marcelk]

Can tuning the evaluation function cause more cut-offs and thus reduce nodes searched for a certain depth?

I've often assumed that to properly tune eval, you'd need fixed depth matches, but this morning I thought that maybe a properly tuned evaluation function merely facilitates a faster search by reducing the nodes searched.

I consistently see this effect in my program. Every time I add
evaluation terms and tune them, the search trees have shrunk.

[rbarreira]

Indeed, better tuned eval is reported to give an enormous reduction of the search tree, with modern search methods.

Is this simply due to the better eval causing the best moves to be found earlier and therefore for example causing less fail-highs at the root, or is it something more complex?

[BubbaTough]

An evaluation that always returns 0 will result in the best move being always found first (ignoring mate and such) and the smallest possible tree. And yet the resulting quality of play is likely poor.

I would say that while good eval often makes for smaller trees than bad eval, optiming tree size instead of chess wins may result in some very bad eval and is not a recommended way to tune a chess engine.

-sam

[hgm]

This is not a realistic example. The point is that any engine will value material in some way. And if it does not have any positional knowledge, it will discover that good moves and positional blunders, which score equal at low depth (because they were not evaluated) will make a huge difference in score at large depth, because an accumulation of positional blunders, although invisible, will eventually translate itself into unavoidable material loss, which is visible.

So it will constanty have to change its PV, because all these easily found 'best' (because equal) moves will almost always at some point run into disastrous fail lows, requiring it to redefine its PV in hitherto unsearched parts of the tree at high depth. This is very expensive.

BTW, no one claimed one should minimize the tree size. The point was that if fixed-depth tests of eval changes show a 5-Elo drop in strength, but the same depth is reached with half the number of nodes at the same NPS, you are more likely to have improved the program by 65 Elo than weakened it by 5. The tree-size difference is a factor undermining the validity of fixed-depth tests.

[PK]

Virtually any program calculates deeper if there's a forcing sequence, i.e. sequence where deviating from the main line causes a big score swing. So it might be that there is a distinct class of eval changes that shrink the tree - namely those that create similar swings, using rather high scores and ocurring with reasonably high frequency.

Probably this hypothesis might be verified easily. Supply TSCP with knowledge about bishop pair (or simply raise its bishop material value) and the tree will become smaller. Supply it with mobility evaluation, which is the exact opposite of "big scores, big leaps" class and the result might even be opposite.