Comparative nodes per second

[bob]

If I recall correctly, just checking the material score actually hurt the elo by something like -20. The lazy eval test hurt by something like -7, and the full eval was a pure wash, no change...

I did leave my other constraints in place of course...

Of course using just material score or even lazy score will hurt if you don't use a margin, because there is a high probability that the static score is above beta if the lazy score is just slightly below beta. Or do you mean that you used lazy eval with the lazy margin subtracted from beta for the comparison? That would be the right way to do it. I'm pretty sure that the reason full eval didn't work for you is just the overhead of the eval, which doesn't apply to us as we need the score anyway for other uses. If you got a zero result despite the cost, then obviously the idea would help if there were no cost.

Yes to the margin idea. But these were pretty fast games. I tried 4 different margins and reported the results for the best one.

Just out of curiosity, what were the best margins for the two cases (lazy and material)?

Since you found that the cost of doing the eval exactly balanced out the benefit, this suggests that you should try using this restriction only when depth is not too low, perhaps above your 4 plies of futility. The cost becomes insignificant if you are far enough from the leaves, but I don't think the same holds for the benefit.

I can give you the margins, but it will be misleading. In Crafty, we have three pieces of the eval.

1. material + a few things, followed by a potential lazy exit using a fairly wide margin.

2. material + a few things + pawn/passed pawn evaluation + things like trapped piece and such. Followed by a second potential lazy exit with a narrower margin.

3. full eval which includes everything above + the individual piece scoring terms and king safety...

The margin for my "lazy eval" would certainly be different (and misleading, hence my hesitance to post it only to have someone try it and fail) because of what my "lazy" eval includes, vs what someone else might include...

I will try the depth limit to see if it changes anything. Just finished the 3 tests at longer games (60s + 1s) and there was no change. Standard 22.5R22 is still the best, R24 was 1 elo worse (but within +/-4 so I consider them equal, and the material only version was -15 worse.

Depth limit tested. Absolutely no difference between eval everywhere and eval everywhere except last 4 plies.

[bob]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

[BubbaTough]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

I expect learning lemming would consider this winning for white, I believe it considers 2 connected passed on the 6th against rook without the king in the area as worth almost a queen, and the lazy eval was fancy enough to not be triggered here. Hannibal has this knowledge as well, but I don't think I set it high enough to quite out weigh the rook in this situation, not sure why. Anyway, I expect a number of engines not in the top tier would consider white to have an edge here (using eval only, no search).

-Sam

[Don]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

Exactly! It's very easy to show positions that computers cannot evaluate correctly without a search. In fact ANY position where a piece is hanging is not evaluated correctly, and that is why we have a quies search. The evaluation is not supposed to evaluate all positions correctly or else we would never need more than a 1 ply search (and only then to select which of the legal moves to play.)

[BubbaTough]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

Exactly! It's very easy to show positions that computers cannot evaluation correctly without a search. In fact ANY position where a piece is hanging is not evaluated correctly, and that is why we have a quies search. The evaluation is not supposed to evaluate all positions correctly or else we would never need more than a 1 ply search (and only then to select which of the legal moves to play.)

I disagree. Well, I agree that evaluation is not 100% accurate in this and most other positions, but that is not saying much as of course Q search and search are not either. What I disagree with is that it is better for eval to consider this position for black because of material concerns. This type of position triggers a known heuristic that human players use, and can be judged without search. Most strong players would look at a glance and say white is winning, and only then consider if there was an escape from check or not and revise their opinion accordingly. Eval is perfectly capable of supporting the simple and accurate heuristic Bob is referring to. The only reason not to have it is if you believe it is a slight elo loser due to the time it takes to calculate. It is a good example of the dangers of lazy eval, which can hinder the application of this heuristic, which I believe was the original reason the position was brought up.

-Sam

[Don]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

Exactly! It's very easy to show positions that computers cannot evaluate correctly without a search. In fact ANY position where a piece is hanging is not evaluated correctly, and that is why we have a quies search. The evaluation is not supposed to evaluate all positions correctly or else we would never need more than a 1 ply search (and only then to select which of the legal moves to play.)

I disagree. Well, I agree that evaluation is not 100% accurate in this and most other positions, but that is not saying much as of course Q search and search are not either. What I disagree with is that it is better for eval to consider this position for black because of material concerns. This type of position triggers a known heuristic that human players use, and can be judged without search. Most strong players would look at a glance and say white is winning, and only then consider if there was an escape from check or not and revise their opinion accordingly. Eval is perfectly capable of supporting the simple and accurate heuristic Bob is referring to. The only reason not to have it is if you believe it is a slight elo loser due to the time it takes to calculate. It is a good example of the dangers of lazy eval, which can hinder the application of this heuristic, which I believe was the original reason the position was brought up.

-Sam

I think you are saying that this exact position is one that an evaluation function should try to evaluate and I don't know why you think we disagree - I didn't say one way or other about whether you should try.

In Komodo we try to do special evaluation when we think we can do it reliably. I don't really know if you can do that position reliably or not. Maybe you can but you didn't specify what class of positions you are talking about. If you mean that EXACT position then it's easy to evaluate, just test for it and score it as a win if you get an exact match.

So you have to specify what specifically is it in that position that you think you can reliably evaluate. Do you mean any position where there is a pawn on the 7th and the opponent doesn't have a pawn on the 7th? What rule covers that STATICALLY without a search? What if the side with the pawn on the 7th can be mated immediately? Now if you impose too many constraints, you make the rule almost useless.

In fact Larry and I decided to work on square of the pawn a while back and we had a very difficult time coming up with RELIABLE rules! We kept adding constraints so that we would not score a win if there was some doubt but after some time we discovered that it's not possible to make this rule perfect, and in fact the harder we tried, the less often the rule could even be used. So this simple rule, to really get right, requires a large decision tree with all sorts of rules - the type of thing you usually don't want to put in a chess program.

Anyway, propose a generally applicable rule that applies to this position and many others and let's see if it's simple or not. What are your rules?

[BubbaTough]

I think you are saying that this exact position is one that an evaluation function should try to evaluate and I don't know why you think we disagree - I didn't say one way or other about whether you should try.

In Komodo we try to do special evaluation when we think we can do it reliably. I don't really know if you can do that position reliably or not. Maybe you can but you didn't specify what class of positions you are talking about. If you mean that EXACT position then it's easy to evaluate, just test for it and score it as a win if you get an exact match.

So you have to specify what specifically is it in that position that you think you can reliably evaluate. Do you mean any position where there is a pawn on the 7th and the opponent doesn't have a pawn on the 7th? What rule covers that STATICALLY without a search? What if the side with the pawn on the 7th can be mated immediately? Now if you impose too many constraints, you make the rule almost useless.

In fact Larry and I decided to work on square of the pawn a while back and we had a very difficult time coming up with RELIABLE rules! We kept adding constraints so that we would not score a win if there was some doubt but after some time we discovered that it's not possible to make this rule perfect, and in fact the harder we tried, the less often the rule could even be used. So this simple rule, to really get right, requires a large decision tree with all sorts of rules - the type of thing you usually don't want to put in a chess program.

Anyway, propose a generally applicable rule that applies to this position and many others and let's see if it's simple or not. What are your rules?

The rule is, 2 connected passed pawns on the 6th (or 7th) can force a queen against a rook if the king is not close enough. Like the square of the pawn rule, you can find situations where it is not a win, but, like the square of the pawn rule, it is winning enough of the time that it is worth using search to find the exceptions rather than to use search to replicate its findings.

Its not uncommon for strong players to sacrifice if they see they can get 2 pawns on the 6th against a rook, without bothering to calculate much further.

-Sam

[bob]

Here is another example for the huge positional score of chess programs(probably also includes IvanHoe that I did not install in my computer and at least it includes robbolito0.09w32 and houdini that gives more than +4 for white at depth 1 when the program cannot see the promotion by search because it is hidden by a lot of checks)

Toga1.2.1a gives 1.17 for black at depth 1 but toga is inferior old program
and I believe that all the programs that are at the playing strength of Rybka3 or higher give a big advantage for white when they search to depth 1 inspite of nominal 6 pawns advantage by black.

[D]7k/PP5p/2PP3p/7p/7p/7p/6r1/K7 b - - 0 1

???
it's black's turn to play in your fen, and it's very obvious that black has an unstoppable passed pawn on h3. the eval should already return a big score for black without even a search!

These positions only illustrate what is already well known, that humans don't evaluate positions the same way that computers do but instead reason about the position. I would remind you that in this case it usually involves not just evaluation but look-ahead - you are looking ahead here to know that there will be a checkmate in 2 moves. You didn't process this in your brain statically.

Computers use a version of the same process and it's called null move search. We will never (in my opinion) have computers match the human brain and it may not even be desirable to try - computers are much different and trying to force them to do things the same way is like trying to force a human to run on all fours because it works for Cheeta's.

In your example of course we know that a computers static evaluation function is not very good at sorting out what is most relevant. That can be improved incrementally with a more sophisticated evaluation function for specific cases but a general solution is probably going to make programs a lot weaker - simply because it will slow the program down enormously and only be useful once in a while. Even the position you show is not a problem for any chess program doing 1 or 2 more ply - although it's easily understood that you can compose arbitrarily more and more difficult cases.

I think that the evaluation function of top programs in this position is very good and they evaluate it even after searching to depth 1 when they cannot see the promotion of white(because black has many checks) as advantage for white.

My evaluation of these kinds of positions is not bad, but Crafty doesn't get this one right without a search, because the black rook prevents a pawn race from being evaluated correctly. In this position with two connected passed pawns on 7th, one could evaluate it correctly since a single piece can't stop the promotion unless the king is right there to help. I'd be interested in seeing a program that does just a one ply search and says "white is winning". Because it is really dangerous to evaluate such positions blindly. If black checks and white escapes check, the eval has basically the same position as it had at the root. Now you have to run down the checks to see if after h2 one of the white pawns can promote and somehow prevent the h2 pawn from promoting.

I think this will likely always be a nominal search + evaluation type of position.

I expect learning lemming would consider this winning for white, I believe it considers 2 connected passed on the 6th against rook without the king in the area as worth almost a queen, and the lazy eval was fancy enough to not be triggered here. Hannibal has this knowledge as well, but I don't think I set it high enough to quite out weigh the rook in this situation, not sure why. Anyway, I expect a number of engines not in the top tier would consider white to have an edge here (using eval only, no search).

-Sam

That's wildly speculative. A good example is WAC 2. Such a bonus can make you find Rxb2 very quickly. But move one pawn, and the white king can now get there in time, and it loses. For example, the e3 pawn to g3 or somewhere irrelevant, and the king has a clear path to help with the two connected passers. Examples like this convinced me to remove such a bonus, although once you get 'em both to the 7th it is pretty easy to do that in evaluation if you think it happens often enough to make a difference...

I removed that code a while back and found zero change in the Elo via cluster testing.

[bob]

I think you are saying that this exact position is one that an evaluation function should try to evaluate and I don't know why you think we disagree - I didn't say one way or other about whether you should try.

In Komodo we try to do special evaluation when we think we can do it reliably. I don't really know if you can do that position reliably or not. Maybe you can but you didn't specify what class of positions you are talking about. If you mean that EXACT position then it's easy to evaluate, just test for it and score it as a win if you get an exact match.

So you have to specify what specifically is it in that position that you think you can reliably evaluate. Do you mean any position where there is a pawn on the 7th and the opponent doesn't have a pawn on the 7th? What rule covers that STATICALLY without a search? What if the side with the pawn on the 7th can be mated immediately? Now if you impose too many constraints, you make the rule almost useless.

In fact Larry and I decided to work on square of the pawn a while back and we had a very difficult time coming up with RELIABLE rules! We kept adding constraints so that we would not score a win if there was some doubt but after some time we discovered that it's not possible to make this rule perfect, and in fact the harder we tried, the less often the rule could even be used. So this simple rule, to really get right, requires a large decision tree with all sorts of rules - the type of thing you usually don't want to put in a chess program.

Anyway, propose a generally applicable rule that applies to this position and many others and let's see if it's simple or not. What are your rules?

The rule is, 2 connected passed pawns on the 6th (or 7th) can force a queen against a rook if the king is not close enough. Like the square of the pawn rule, you can find situations where it is not a win, but, like the square of the pawn rule, it is winning enough of the time that it is worth using search to find the exceptions rather than to use search to replicate its findings.

Its not uncommon for strong players to sacrifice if they see they can get 2 pawns on the 6th against a rook, without bothering to calculate much further.

-Sam

I think it is dangerous to assume a win and let the search find a refutation if one exists. A deep / wide search can use (still) the horizon effect to its advantage here to cause embarrassing moves in real games. I trust the search more than the eval when we are talking about races, checks, interference squares and such. I suppose it boils down to how much error, and where, are you willing to tolerate?

[BubbaTough]

I think you are saying that this exact position is one that an evaluation function should try to evaluate and I don't know why you think we disagree - I didn't say one way or other about whether you should try.

In Komodo we try to do special evaluation when we think we can do it reliably. I don't really know if you can do that position reliably or not. Maybe you can but you didn't specify what class of positions you are talking about. If you mean that EXACT position then it's easy to evaluate, just test for it and score it as a win if you get an exact match.

So you have to specify what specifically is it in that position that you think you can reliably evaluate. Do you mean any position where there is a pawn on the 7th and the opponent doesn't have a pawn on the 7th? What rule covers that STATICALLY without a search? What if the side with the pawn on the 7th can be mated immediately? Now if you impose too many constraints, you make the rule almost useless.

In fact Larry and I decided to work on square of the pawn a while back and we had a very difficult time coming up with RELIABLE rules! We kept adding constraints so that we would not score a win if there was some doubt but after some time we discovered that it's not possible to make this rule perfect, and in fact the harder we tried, the less often the rule could even be used. So this simple rule, to really get right, requires a large decision tree with all sorts of rules - the type of thing you usually don't want to put in a chess program.

Anyway, propose a generally applicable rule that applies to this position and many others and let's see if it's simple or not. What are your rules?

The rule is, 2 connected passed pawns on the 6th (or 7th) can force a queen against a rook if the king is not close enough. Like the square of the pawn rule, you can find situations where it is not a win, but, like the square of the pawn rule, it is winning enough of the time that it is worth using search to find the exceptions rather than to use search to replicate its findings.

Its not uncommon for strong players to sacrifice if they see they can get 2 pawns on the 6th against a rook, without bothering to calculate much further.

-Sam

I think it is dangerous to assume a win and let the search find a refutation if one exists. A deep / wide search can use (still) the horizon effect to its advantage here to cause embarrassing moves in real games. I trust the search more than the eval when we are talking about races, checks, interference squares and such. I suppose it boils down to how much error, and where, are you willing to tolerate?

I think it really only boils down to which is more accurate, more often. If you look up rook vs. 2 connected passed on the 6th in Dvoretsky's endgame manual, you will see all sorts of tricks showing the rook finding a draw against the pawns. But these are really just exceptions to the rule. In a vast majority of cases the pawns win. VAST majority. And I find it embarrassing in real games to miss these, which is a risk if you prune a lot. Any statement like "I suppose it boils down to how much error, and where, are you willing to tolerate?" can always be turned on it head. How much error in your Eval's assumption that the rook is better than the pawns are YOU willing to tolerate, given the embarrassing moves relying on your search may lead you to?

-Sam