If you can fail lower or higher by not doing lazy eval, you will get more cutoffs.bob wrote:How can it introduce an efficiency. You NEVER use a lazy cutoff test that is not 100% accurate, because you know the largest possible positional score for each piece type. If you have no bishops, there can be no score for bishops, the lazy eval cutoff window shrinks.lucasart wrote:Even the Bruce Moreland no risk way iintroduces inefficiency with fail soft. I use fail soft and no lazy eval: that's what works best for me. And I conjecture that lazy eval will not scale, like anything that compromises accuracy for a pesky speed gain(which gains less at LTC...)
What would you do differently
Moderators: hgm, Rebel, chrisw
-
- Posts: 3232
- Joined: Mon May 31, 2010 1:29 pm
- Full name: lucasart
Re: lazy eval
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
-
- Posts: 27790
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: lazy eval
Usually investing in things that might happen is a losing deal in Chess engines. You also don't go on searching moves in a node after you find one that beats beta, just because a still higher score for that node might give you more cutoffs in case of transpositions. And because of the lazy nature of alpha-beta pruning, scores tend to rapidly converge towards the window when you get closer to the root, even if you would produce tighter bounds in eval.
-
- Posts: 589
- Joined: Tue Jun 04, 2013 10:15 pm
Re: lazy eval
Don't you think that Crafty is stuck at a local optimum, evaluation-wise, with such approach? Have you given your weights tuner all the room it asks?bob wrote:I don't view it quite the same. In Crafty, we have several sorts of eval terms.mvk wrote:I see lazy eval as a pitfall. If you find it helps a lot, you are searching the wrong part of the tree and should work on that first. And then it keeps working against you when tuning the evaluation weights, and the tree shaping in general because of the lost fidelity: every change in the right direction now first must overcome a barrier raised by this premature optimisation. Development-wise, it therefore tends to keep you locked in a local optimum of program strength. I still have it in the code, but with huge bounds so it is effectively off. This might be different if you're within 30 elo of the top and must scrape the barrel.jdart wrote:Yes, when I had lazy eval, I passed in the bounds.
But if you call it multiple times with different bounds, you can't cache and re-use the result. And this negates some of the benefit of lazy eval. Now I don't use it but the result of eval is cached (and stored in the hashtable).
--Jon
1. Those that produce big score changes
2. Those that produce small score changes but are computationally expensive.
[Account deleted]
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: lazy eval
It is trivial to turn off. What you are asking is, "is it safe to assume that not all eval terms are going to reach their max or min in the same position?" You accept some risk, you gain some speed. To date, getting rid of it has ALWAYS hurt by at least 10-20 Elo. I don't do CLOPS type tuning at all, I've never had any luck with either that or the annealing stuff I did with Cozzie years back. Never got any results that tested better than or even as good as the normal hand-tuned eval terms.mvk wrote:Don't you think that Crafty is stuck at a local optimum, evaluation-wise, with such approach? Have you given your weights tuner all the room it asks?bob wrote:I don't view it quite the same. In Crafty, we have several sorts of eval terms.mvk wrote:I see lazy eval as a pitfall. If you find it helps a lot, you are searching the wrong part of the tree and should work on that first. And then it keeps working against you when tuning the evaluation weights, and the tree shaping in general because of the lost fidelity: every change in the right direction now first must overcome a barrier raised by this premature optimisation. Development-wise, it therefore tends to keep you locked in a local optimum of program strength. I still have it in the code, but with huge bounds so it is effectively off. This might be different if you're within 30 elo of the top and must scrape the barrel.jdart wrote:Yes, when I had lazy eval, I passed in the bounds.
But if you call it multiple times with different bounds, you can't cache and re-use the result. And this negates some of the benefit of lazy eval. Now I don't use it but the result of eval is cached (and stored in the hashtable).
--Jon
1. Those that produce big score changes
2. Those that produce small score changes but are computationally expensive.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: lazy eval
You will have to run that one by me again. Fail-soft doesn't guarantee anything, and in general, you cut off whether the score is == beta, or > beta, how much doesn't matter. Only place I have EVER seen fail-soft useful is in mtd(f) since anything that gets you closer to the true score with fewer re-searches is a good thing...lucasart wrote:If you can fail lower or higher by not doing lazy eval, you will get more cutoffs.bob wrote:How can it introduce an efficiency. You NEVER use a lazy cutoff test that is not 100% accurate, because you know the largest possible positional score for each piece type. If you have no bishops, there can be no score for bishops, the lazy eval cutoff window shrinks.lucasart wrote:Even the Bruce Moreland no risk way iintroduces inefficiency with fail soft. I use fail soft and no lazy eval: that's what works best for me. And I conjecture that lazy eval will not scale, like anything that compromises accuracy for a pesky speed gain(which gains less at LTC...)
-
- Posts: 589
- Joined: Tue Jun 04, 2013 10:15 pm
Re: lazy eval
When you tried to auto-tune the weights, did you gauge progress against a version with lazy evaluation, or relative to a version without such cut-offs? In the first case, you might have been tuning against the wall of the local optimum.bob wrote:It is trivial to turn off. What you are asking is, "is it safe to assume that not all eval terms are going to reach their max or min in the same position?" You accept some risk, you gain some speed. To date, getting rid of it has ALWAYS hurt by at least 10-20 Elo. I don't do CLOPS type tuning at all, I've never had any luck with either that or the annealing stuff I did with Cozzie years back. Never got any results that tested better than or even as good as the normal hand-tuned eval terms.mvk wrote:Don't you think that Crafty is stuck at a local optimum, evaluation-wise, with such approach? Have you given your weights tuner all the room it asks?bob wrote:I don't view it quite the same. In Crafty, we have several sorts of eval terms.mvk wrote:every change in the right direction now first must overcome a barrier raised by this premature optimisation. Development-wise, it therefore tends to keep you locked in a local optimum of program strength. I still have it in the code, but with huge bounds so it is effectively off. This might be different if you're within 30 elo of the top and must scrape the barrel.
1. Those that produce big score changes
2. Those that produce small score changes but are computationally expensive.
[Account deleted]
-
- Posts: 893
- Joined: Mon Jan 15, 2007 11:23 am
- Location: Warsza
Re: lazy eval
Perhaps an aspiration window widened in several stages (more than two or three) might behave more like MTD(f) and benefit from fail-soft + eager eval (is it a correct antonym?)
Anyhow, my problem with lazy eval is different: statistically it helps my engine a lot, but at the same time it causes it to misevaluate positions with really high king safety scores. Increasing the margin (going towards Bruce Moreland's optimum) solves this problem but decreases Elo. Doing king safety before lazy eval cutoff and mobility afterwards would mean looping through the pieces twice and generating similar bitboards twice (added complexity, negating some of speed benefits). Perhaps foregoing lazy eval together with its benefits is a better decision for a casual programmer.
Anyhow, my problem with lazy eval is different: statistically it helps my engine a lot, but at the same time it causes it to misevaluate positions with really high king safety scores. Increasing the margin (going towards Bruce Moreland's optimum) solves this problem but decreases Elo. Doing king safety before lazy eval cutoff and mobility afterwards would mean looping through the pieces twice and generating similar bitboards twice (added complexity, negating some of speed benefits). Perhaps foregoing lazy eval together with its benefits is a better decision for a casual programmer.
Pawel Koziol
http://www.pkoziol.cal24.pl/rodent/rodent.htm
http://www.pkoziol.cal24.pl/rodent/rodent.htm
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: lazy eval
I do the aspiration widening in several stages. Here is an example from current Crafty:PK wrote:Perhaps an aspiration window widened in several stages (more than two or three) might behave more like MTD(f) and benefit from fail-soft + eager eval (is it a correct antonym?)
Anyhow, my problem with lazy eval is different: statistically it helps my engine a lot, but at the same time it causes it to misevaluate positions with really high king safety scores. Increasing the margin (going towards Bruce Moreland's optimum) solves this problem but decreases Elo. Doing king safety before lazy eval cutoff and mobility afterwards would mean looping through the pieces twice and generating similar bitboards twice (added complexity, negating some of speed benefits). Perhaps foregoing lazy eval together with its benefits is a better decision for a casual programmer.
Code: Select all
23-> 9.39/54.19 7.90 73. Rd5 Rb2 74. Be8+ Kc7 75. e5 Rf2+ 76.
Kg3 Rf8 77. Bb5 Rg8+ 78. Kf4 Rg6 79. Bd7
Rh6 80. Kg5 Rh8 81. Bf5 Kc6 82. Rd7 Rg8+
83. Kf6 Rf8+ 84. Rf7 Rh8 85. e6 Rh6+ 86.
Kg5 Rh2 87. Bg4
24 9.56/54.19 ++ 73. Rd5! (>+8.06)
24 9.73/54.19 ++ 73. Rd5! (>+8.22)
24 9.91/54.19 ++ 73. Rd5! (>+8.54)
24 10.13/54.19 ++ 73. Rd5! (>+9.18)
24 11.04/54.19 10.23 73. Rd5 Rb2 74. Be8+ Kc7 75. e5 Rf2+ 76.
Kg3 Rf8 77. Bb5 Rg8+ 78. Kf4 Rg6 79. Bd7
Rh6 80. Bg4 Rh7 81. e6 Rg7 82. Rd3 Re7
83. Kg5 Kc6 84. Rd7 Rxe6 85. Bxe6 Kc5 86.
Rd5+ Kc4 87. Rf5+ Kd3
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: lazy eval
No lazy for that. I just never got anything better from auto-tuning. Got some pretty strange looking values, but they were slightly worse. I believe the best I have seen ever was like -5-6 Elo. Cozzie had similar results last time we talked. He thought his annealing was going to be "the greatest thing ever in computer chess" but then along came a reality check.mvk wrote:When you tried to auto-tune the weights, did you gauge progress against a version with lazy evaluation, or relative to a version without such cut-offs? In the first case, you might have been tuning against the wall of the local optimum.bob wrote:It is trivial to turn off. What you are asking is, "is it safe to assume that not all eval terms are going to reach their max or min in the same position?" You accept some risk, you gain some speed. To date, getting rid of it has ALWAYS hurt by at least 10-20 Elo. I don't do CLOPS type tuning at all, I've never had any luck with either that or the annealing stuff I did with Cozzie years back. Never got any results that tested better than or even as good as the normal hand-tuned eval terms.mvk wrote:Don't you think that Crafty is stuck at a local optimum, evaluation-wise, with such approach? Have you given your weights tuner all the room it asks?bob wrote:I don't view it quite the same. In Crafty, we have several sorts of eval terms.mvk wrote:every change in the right direction now first must overcome a barrier raised by this premature optimisation. Development-wise, it therefore tends to keep you locked in a local optimum of program strength. I still have it in the code, but with huge bounds so it is effectively off. This might be different if you're within 30 elo of the top and must scrape the barrel.
1. Those that produce big score changes
2. Those that produce small score changes but are computationally expensive.