The largest Eval Jump I've ever seen from any program!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Uri Blass
Posts: 10102
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: The largest Eval Jump I've ever seen from any program!

Post by Uri Blass »

pijl wrote:
Uri Blass wrote:I consider it as a bug because a program should never evaluate it as +14 in the first place.
Is it not that simple. As you undoubtedly know, all of the evaluation of chess program consists of many patterns which all contribute to the evaluation score of the program. One pattern that is present is all of the stronger chess programs is the quadrant rule in pawn endings. Some programs have this rule in a crude form (i.e. if king is outside the quadrant, the pawn is an unstoppable passer and if the opponent doesn't have one, add a big bonus) and rely on the search to correct it in the few cases where it is wrong, while others also try to assess means to stop the passer in different ways and are more hesitant to add a big bonus when the opponent also has passers, especially when they are more advanced. One of those methods is demonstrated by a key position that Rybka may have based its big bonus on:
[D]8/8/3P1kP1/4p3/1p5p/8/8/7K w - -
The pawn on b4 is the 'unstoppable' in this case, but white can create a more advanced one by moving either of its passers and the king will be out of the quadrant for one of them.
CTD gives a static evaluation for this position of -8.554, but sees quickly that this is incorrect when the search is started.
The Baron has a more refined method of using the quadrant rule when there are more than one passers for one side. As a result, the static evaluation score of the Baron for this position is just -1.64.
The method used is pretty simple:
Instead of applying the quadrant rule to just one pawn, I 'and' the quadrant bitboards of all passers of one side together and test that with the king. Additionally, if the resulting bitboard does not contain all the promotion squares, I know this side will have an 'unstoppable' passer too.

The remaining question is now why it takes so long for Rybka to see that Ne3 is not really the best move. I can only imagine that this is due to the pruning logic in Rybka.
Richard.
I have clearly simpler and worse rules in movei.
I simply do not evaluate pawn as unstoppable if the opponent has more advanced pawns.

I do not claim that it is easy to have correct rules.
The point is that a very big bonus and I talk about +10 and not about +2 is something that you should not give if you are not sure that there are no exceptions.

Uri
pijl

Re: The largest Eval Jump I've ever seen from any program!

Post by pijl »

Uri Blass wrote: I have clearly simpler and worse rules in movei.
I simply do not evaluate pawn as unstoppable if the opponent has more advanced pawns.
I've tried that too, but in the end giving the bonus gained more than not giving prevented losses.
Uri Blass wrote:I do not claim that it is easy to have correct rules.
The point is that a very big bonus and I talk about +10 and not about +2 is something that you should not give if you are not sure that there are no exceptions.
Yes, this is usually prudent. I'm quite careful with big bonusses myself, but in the case of unstoppable passers it is usually better to given them anyway and let the search sort out the exceptions that are usually spotted with a shallow search anyway.
Pruning may be a spoiler here though ...
Richard.
pijl

Re: The largest Eval Jump I've ever seen from any program!

Post by pijl »

bob wrote:Maybe I didn't quite follow, but something I have played with recently to address just this case was that if one side has two passed pawns, and both are stopped by the king, then I go a little further and ask "if either pawn were my king, could it stop the other one?" If the answer is no, then the pawns are unstoppable since if I capture one, the other one runs.
We may have implemented the same thing in a slightly different way, but just to make sure, let me explain a bit further.
In the position presented, there are pawns on d6 and g6, with white to move.
The quadrant of the d6 pawn is basically the square d6-f8, while the square of g6 is g6-e8. Obviously, the king has to be in both squares to have a chance to stop both pawns.
Now suppose we're sacrificing one pawn, so the king has to really stop it by blocking or capturing it, ultimately on the promotion square. Can we force it to move out of the quadrant of the other pawn? If the promotion square of the sacrificed pawn is outside the common quadrant area we can. So the relatively simple method of and-ing both quadrant bitboards, and checking whether the promotion squares and the king are all within this combined and shrunk bitboard, provides a simple test on the danger of the pawns which may spot problems a few plies earlier.
The only reason that I did not add it to CTD's evaluation is a simple one: It did not have any priority. In the end, positions like this are exceptions and usually there are bigger problems to solve :-)
Richard.