Lazy eval

[bob]

i had many times experimented with and without lazy eval in the past, and i was danced between to use or not LE.

with LE you become of course a speed up , but on the other side if you using very agressive pruning and LMR in search this doesn' t help you to reduce the tree size on the end of search. So my mind is you have to do more nodes to search with LE, but without you can pruning more moves in search if you eval is more accurate. The other dangarous is if you use a complex king safety eval and return a margin back for pruning decisions, so this can not return back because you leaf the eval before king safety is done.

So i decide now not to use Lazy Eval ! .

Why does lazy eval affect pruning? I'd think the usual test is in relation to alpha/beta window, and a lazy exit should return one of those two bounds, or you could return something outside those bounds if you want to experiment...

[mcostalba]

Do you remember how much of a NPS gain you got from lazy eval? If not we could test it with your patch.

I have pushed the lazy_eval branch that I have resurrected from the last spring and rebased above current master:

https://github.com/mcostalba/Stockfish/ ... /lazy_eval

I did some experiment and some tweak but with no success.

[Engin]

its seems that is doing, because the full eval without LE is more accurate and decide a better pruning of moves and reduce the tree size on the end.
i believe that. with LE score returns a speculative score.

[bob]

its seems that is doing, because the full eval without LE is more accurate and decide a better pruning of moves and reduce the tree size on the end.
i believe that. with LE score returns a speculative score.

Certainly it is speculative. But HOW speculative? The q-search by itself is HIGHLY selective, after all, it only considers "good" captures, where in chess, the best move is generally NOT a capture.

[Engin]

yes, eval scores are important for pruning moves in my engine:

in quiesce i use futility pruning like this for captures:

futility_score = eval_score + capture_value ;

if(futility_score < alpha) .... prune

and in the main pvs search only if not in check and for quiet moves :

futility_score = eval_score + futility_margin

if(futlity_score < beta) .... prune


may that is the reason why with LE not work ?

[Engin]

..and i am using the eval score in pvs search before searching moves to razoring, static eval before null move pruning and null move pruning decide if eval_score >=beta prune

...and eval_score is also improtant for internal iterative search in non-pv nodes too.

[Evert]

Our program spends more time on eval than most, I think it's over half the time. So lazy eval should be a big win. But it doesn't seem to kick in nearly as often as it must be doing in other programs. The puzzle is whether it's due to other things we do or to wrong implementation of the idea.

Well, I guess it's not obvious that lazy evaluation would always help.
Suppose that you have a perfect evaluation function (if you did, you could just drop the search altogether, but let's not follow this line of thought that far). Any lazy evaluation you apply to the perfect evaluation function makes it worse, which means your program will play more poorly.
Clearly, there must be a point where an imperfect evaluation function (with a search) is so good that any early exits by lazy evaluation hurt more than you can compensate for by extra speed. The question then, of course, is whether you think your evaluation function is at that point. That's an entirely separate discussion and I'm not qualified to formulate an opinion either way (but I would personally be surprised if any chess program has such a good evaluation function).

Independent of whether the evaluation is at this point or not, there is at least one possible reason why lazy evaluation will not work for you, there might be others. It just doesn't necessarily mean that there's anything wrong if it doesn't work.

[Milos]

..and i am using the eval score in pvs search before searching moves to razoring, static eval before null move pruning and null move pruning decide if eval_score >=beta prune

Nobody in their right mind would use lazy eval in PV nodes...

[Engin]

my pvs search include pv and non-pv nodes in one.

i do eval only in non check nodes, but prune only in non-pv nodes

[lkaufman]

Do you remember how much of a NPS gain you got from lazy eval? If not we could test it with your patch.

I have pushed the lazy_eval branch that I have resurrected from the last spring and rebased above current master:

https://github.com/mcostalba/Stockfish/ ... /lazy_eval

I did some experiment and some tweak but with no success.

OK, we tried a more aggressive version of lazy and we did get the big NPS gains of 10% or more. However the number of nodes increased and the quality went down, so whatever margin we use the tradeoff is bad. So the mystery is why lazy eval works so well for Critter and for all the Rybka and Ippolit-related programs, but fails in both Stockfish and Komodo? Both SF and Komodo use pretty aggressive king safety scores, but I don't think they are drastically higher (when scaled based on the average value of an extra pawn) than these Ippo- related programs, at least not so drastically higher as to turn a free 10% speedup in Critter into a clear net loss in both SF and Komodo. Can you think of anything that would make the ippo programs behave so differently from SF in this matter? Or could we both be missing some crucial implementation detail?