Lazy eval

[tvrzsky]

OK, you have almost convinced me that lazy eval just won't work in a program with big eval scores for king safety and pawn structure. However, it seems to me that we should still get the usual gain in nodes per second, even if the increased size and inaccuracy of the tree makes it a bad deal. But we only get at most 2-3% more nodes per second, while at least Critter gets 10% or so. Why should our nodes per second gain depend much on the eval details?

Did you make some statistics how often is your eval really lazy and did you measure how many CPU cycles you can save with it? This should give you answer after all?

Yes, we took some statistics, and they indicated that it didn't kick in very often unless the margin was very small. I'm just not sure why it didn't kick in more often.

And this is an answer I guess. Savings should be simple function of complexity of evaluation and frequency of lazy eval. Perhaps you need to find why your conditions for taking lazy eval don't work.
Or maybe you don't have too many imbalanced positions in your search tree? I. E. your search is already smart enough to avoid positions where lazy eval could be taken?

[lkaufman]

Yes, we took some statistics, and they indicated that it didn't kick in very often unless the margin was very small. I'm just not sure why it didn't kick in more often.

And this is an answer I guess. Savings should be simple function of complexity of evaluation and frequency of lazy eval. Perhaps you need to find why your conditions for taking lazy eval don't work.
Or maybe you don't have too many imbalanced positions in your search tree? I. E. your search is already smart enough to avoid positions where lazy eval could be taken?

Yes, I suppose this is correct. However, I think Critter and other top programs also prune unbalanced positions very severely near the end, and yet it works for Critter reasonably well. How about Crafty, does it also prune severely on the last few plies? Bob says it works fine in Crafty.

[bob]

[quote="bobQuestion is, why doesn't it work? The idea is speed. If you do it well, you should be able to see a speed gain that more than offsets the minor accuracy loss. I've been using it in every program I've written since the 70's, The "heavier" the eval, the greater the potential gain... One can even do a "staged" lazy eval with multiple exits, rather than just one... Each exit placed before the next "heavier" piece of the evaluation...

I take it that Crafty does get a decent speedup for lazy eval. Do you get about the same as the 10% figure I quote for Critter, or perhaps more? We can't get more than 2% with any reasonable margin, clearly not enough to use it. The question is whether this is due to redundancy with other pruning techniques, or to some implementation detail? Of course you won't know as you don't work on Komodo, but perhaps you can suggest a way to tell. Sure we do a lot of heavy pruning near the leaf nodes, but so does Critter and it gets a very nice speedup from lazy eval.[/quote]

I'll try to test later tonight and report the speedup... I'll have to go back to look at exactly what we are doing... we have changed this stuff several times, from one lazy exit, to multiple, and such...

[Don]

Do you get about the same as the 10% figure I quote for Critter

Now it's my time to ask. How can you know the speed up of lazy eval for Critter.

Do you have the Critter sources ?

Richard sent me the sources long ago. I made a version that turns off lazy evaluation for testing.

Don

[bob]

[quote="bobQuestion is, why doesn't it work? The idea is speed. If you do it well, you should be able to see a speed gain that more than offsets the minor accuracy loss. I've been using it in every program I've written since the 70's, The "heavier" the eval, the greater the potential gain... One can even do a "staged" lazy eval with multiple exits, rather than just one... Each exit placed before the next "heavier" piece of the evaluation...

I take it that Crafty does get a decent speedup for lazy eval. Do you get about the same as the 10% figure I quote for Critter, or perhaps more? We can't get more than 2% with any reasonable margin, clearly not enough to use it. The question is whether this is due to redundancy with other pruning techniques, or to some implementation detail? Of course you won't know as you don't work on Komodo, but perhaps you can suggest a way to tell. Sure we do a lot of heavy pruning near the leaf nodes, but so does Critter and it gets a very nice speedup from lazy eval.

I'll try to test later tonight and report the speedup... I'll have to go back to look at exactly what we are doing... we have changed this stuff several times, from one lazy exit, to multiple, and such...[/quote]

I have two lazy eval exits in my evaluate.c code. I disabled both, and ran three test positions for 30 seconds each. The starting position, a middlegame position, and fine 70 which is a worst-case comparison probably in there are nothing but kings and pawns...

The first output for each pair is the no lazy eval code, the second with with normal lazy eval:

log.001: time=30.08 mat=0 n=46031404 fh=91% nps=1.5M
log.002: time=30.06 mat=0 n=53408994 fh=91% nps=1.8M
improvement: 7.4M additional nodes searched, 16% faster.

log.001: time=30.05 mat=0 n=68478030 fh=95% nps=2.3M
log.002: time=30.11 mat=0 n=91888687 fh=95% nps=3.1M
improvement: 23.4M additional nodes searched, 34% faster

log.001: time=30.10 mat=1 n=129214534 fh=93% nps=4.3M
log.002: time=30.01 mat=1 n=121312909 fh=87% nps=4.0M
improvement: 8M additional nodes searched, 6% faster.

All searches were for 30 seconds, so more nodes is better

In the middlegame, it is a pretty big savings.

[Rebel]

SF has very sophisticated (and expensive) king safety evaluation which can return high scores. If we used lazy eval, we would effectively get less accurate king safety eval.[/list]

That's indeed THE danger of LE, if your evaluation returns high scores bare LE becomes a curse. It's solvable, see:

http://www.top-5000.nl/authors/rebel/chess840.htm

Pick Lazy Eval from the right menu.

You should really get a good speed-up improvement when using the eval score of RD1.

[Rebel]

Have you tried Ed Schröder's way? http://www.top-5000.nl/authors/rebel/ch ... AZY%20EVAL

You beat me to it

[Rebel]

Actually, that description seems to match what we tried very closely! Ed reports a 3.2 speedup (i.e. 220%) while we get barely over 2%! So it works 100 times better for him than for us, even though we both have complex king safety with large possible scores. Very, very strange!!

If you have ProDeo change the LE parameter:

[Chess Knowledge = 100] to 500

LE is more or less deactivated then.

Then check the differences between 100 and 500.

I quickly ran 2 positions:

1) 100: 22M nodes vs 500: 47M nodes
2) 100: 21M nodes vs 500: 55M nodes

[brianr]

FWIW lazy eval speeds things up by about 16% for Tinker from the opening position.

I also have potentially large eval scores related to pawns and king safety, so I simply group the other smaller, more positional and mobility related terms at the end of eval and only apply "lazy" to them.

In testing, lazy eval margins have been between 0.25 and up to 2.00
It is currently 1.30 with the newer mobility.

[tpetzke]

All searches were for 30 seconds, so more nodes is better

Is it really possible to state that ? Should the quality of the result not also influence the statement about the final outcome.

Even if both searches return the same move after 30 sec, maybe the slower search also required less nodes to find this move.

Thomas...