Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.lkaufman wrote:Well, Critter gets about 10% for a 60 cp margin, so not all programs get as much as you say. But our figures are indeed suspiciously low. Still, it's possible that other things we do are so similar to lazy eval that this could be the explanation. We'll have to investigate further.Milos wrote:This is clearly impossible. You must have an error in implementation.lkaufman wrote:But with margins of around half a pawn, we only get about a 2% speedup (roughly 4% more nps but a 2% increase in nodes) and our quality clearly suffers too much for that. Can you check what speedup you get, and with what margins?
The usual speed saving is 10% for 2-3 pawns margin. For under 100cp margin you should get well over 15%.
Lazy eval
Moderators: hgm, Rebel, chrisw
-
- Posts: 481
- Joined: Thu Apr 16, 2009 12:00 pm
- Location: Slovakia, EU
Re: Lazy eval - test results
-
- Posts: 1471
- Joined: Tue Mar 16, 2010 12:00 am
Re: Lazy eval - test results
What a coincidence, Houdini does exactly the same...rvida wrote:Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.
LOL.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Lazy eval - test results
Do you remember how much of a NPS gain you got from lazy eval? If not we could test it with your patch.mcostalba wrote:For a very selective search you may want to have accurate eval info to push pruning to the limits. In case of Crafty, where the pruning is far lower I guess given the 300 ELO gap or so, my guess is the speed up plays a factor more than accurate eval. Another point are the spikes in king evaluations, present in SF and, as you mentioned also in Komodo, these are filtered out with lazy eval, but play an important role for pruning decision.lkaufman wrote: I note that SF rejected lazy eval. So at least one other top program failed to demonstrate a benefit from it. Why didn't it help Stockfish? I'm sure they would not have rejected a 10% nearly-free speedup.
I still should have the SF lazy eval patch somewhere, I can post it if someone is interested in experimenting with this.
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: Lazy eval - test results
If you use a pure positional value that's a bit too restrictive. I would lower the margin for that condition and use positional-PST instead of just positional.Houdini wrote:What a coincidence, Houdini does exactly the same...rvida wrote:Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.
LOL.
Also the margin should be dependent on if the position is quiet or not.
-
- Posts: 1471
- Joined: Tue Mar 16, 2010 12:00 am
Re: Lazy eval - test results
You may or may not be right, but that wasn't my point.Milos wrote:If you use a pure positional value that's a bit too restrictive. I would lower the margin for that condition and use positional-PST instead of just positional.Houdini wrote:What a coincidence, Houdini does exactly the same...rvida wrote:Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.
LOL.
Also the margin should be dependent on if the position is quiet or not.
It's just funny to see how much of Houdini ideas have already made it into other engines...
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: Lazy eval - test results
Don quoted the value of 150 in qsearch for Critter, which I took to mean 150/256 or about 0.6 since you use 256 for a pawn. Was this 150 actually in centipawns, i.e. 384 on your scale? If so it is amazing that you get 10% for lazy with such a large margin.rvida wrote:Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.lkaufman wrote:Well, Critter gets about 10% for a 60 cp margin, so not all programs get as much as you say. But our figures are indeed suspiciously low. Still, it's possible that other things we do are so similar to lazy eval that this could be the explanation. We'll have to investigate further.Milos wrote:This is clearly impossible. You must have an error in implementation.lkaufman wrote:But with margins of around half a pawn, we only get about a 2% speedup (roughly 4% more nps but a 2% increase in nodes) and our quality clearly suffers too much for that. Can you check what speedup you get, and with what margins?
The usual speed saving is 10% for 2-3 pawns margin. For under 100cp margin you should get well over 15%.
-
- Posts: 4366
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Lazy eval
I've always used it. I use a pretty large margin for evaluating a score lazily - i.e. it has to be far above beta, or below alpha, to skip the rest of the eval. Once in a while I turn on a debug mode that prints out the FENs of any positions that would have been inaccurately scored with lazy eval - in other words, would have caused an early cutoff but if the full eval was done, the score would have been below beta (for a high cutoff) or above alpha (for a low cutoff). Usually that's a small number, so I am being conservative and not accepting a lot of error.
As noted, the problem is king safety. For me that's an expensive calculation, so I'd like to avoid it with lazy eval. But the king safety scores can be relatively large. Hence the need for a high margin, which reduced the effectiveness of lazy eval.
--Jon
As noted, the problem is king safety. For me that's an expensive calculation, so I'd like to avoid it with lazy eval. But the king safety scores can be relatively large. Hence the need for a high margin, which reduced the effectiveness of lazy eval.
--Jon
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: Lazy eval - test results
This particular idea is something that is below 5Elo and practically unmeasurable.Houdini wrote:You may or may not be right, but that wasn't my point.
It's just funny to see how much of Houdini ideas have already made it into other engines...
I guess some ppl copy ideas mechanically following the principle if it doesn't hurt it helps. Others however, require to see measurable benefit.
Different ppl, different approaches .
P.S. I wouldn't be too worried. Only 2 ppl capable and willing to RE for ideas (and with their own strong engines) have done it so far . Others just go around and ask questions .
P.P.S You also made it too easy since everyone has an exact source of Houdini beta .
-
- Posts: 1471
- Joined: Tue Mar 16, 2010 12:00 am
Re: Lazy eval - test results
I'm not worried. Like I said before, imitation is the sincerest form of flattery.Milos wrote:This particular idea is something that is below 5Elo and practically unmeasurable.Houdini wrote:You may or may not be right, but that wasn't my point.
It's just funny to see how much of Houdini ideas have already made it into other engines...
I guess some ppl copy ideas mechanically following the principle if it doesn't hurt it helps. Others however, require to see measurable benefit.
Different ppl, different approaches .
P.S. I wouldn't be too worried. Only 2 ppl capable and willing to RE for ideas (and with their own strong engines) have done it so far . Others just go around and ask questions .
P.P.S You also made it too easy since everyone has an exact source of Houdini beta .
I just notice that Houdini's specific implementation of an original idea is being presented by an acclaimed author, without giving any credit.
Unless this has been in Critter from before version 1.0 (Richard RE'd Houdini 1.5 between Critter 0.90 and 1.0), and we've just ended up by a great coincidence using exactly the same code?!
Robert
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Lazy eval - test results
My LE-margin is 0.50 for the leafs and QS.rvida wrote: Not sure where did you get the quoted 60cp margin. Critter uses 150 cp lazy margin in qsearch and 300 cp margin at interior nodes. Also, when the absolute value of positional component from ply-1 eval is greater than 150 cp, lazy eval is not used.
Here is what I do.
ISCORE = incremental update score of material + PST's
SCORE = score of EVAL
RDx = remaining depth
In EVAL I stack (ISCORE-SCORE) + LE-margin (0.50)
Level=3 Ply
1. e4 e5 2. Nf3
Ar RD0 (Nf3) before make_move:
1. Update ISCORE
2. SCORE = ISCORE + stacked score of RD1 (e5)
3. if (alpha >= SCORE) allow LE and prune; else EVAL
Same for the whole QS.
That's it except for the usual troublemakers so I exclude LE when:
1. King in Check
2. Move gives a check (you probably need to write some new code)
3. Extreme cases like an unstoppable pawn (+800) at RD1, I stack those also in EVAL.