Another Impossible Endgame For Engines

[lmader]

What Stephen is trying to say, is that engines are clueless in this position...

Yes, you are correct, the high level point is exactly as you stated.

I was just responding to the business about how the evaluation was being interpreted. It seemed as though he was saying that a non-zero eval in the endgame meant to him that the program was declaring a decisive outcome, when in fact it just means that the engine is clueless, as you point out.

[Stephen Ham]

What Stephen is trying to say, is that engines are clueless in this position...

Yes, you are correct, the high level point is exactly as you stated.

I was just responding to the business about how the evaluation was being interpreted. It seemed as though he was saying that a non-zero eval in the endgame meant to him that the program was declaring a decisive outcome, when in fact it just means that the engine is clueless, as you point out.

Hello Lar and Miguel,

Miguel is correct and I'm glad he communicated the point clearly to you. But Lar, my communication was already clear starting with the title of my post.

The reason I posted was to provide evidence of the cluelessness of chess engines.

All the best,
Steve

[jwes]

What Stephen is trying to say, is that engines are clueless in this position...

Yes, you are correct, the high level point is exactly as you stated.

I was just responding to the business about how the evaluation was being interpreted. It seemed as though he was saying that a non-zero eval in the endgame meant to him that the program was declaring a decisive outcome, when in fact it just means that the engine is clueless, as you point out.

Hello Lar and Miguel,

Miguel is correct and I'm glad he communicated the point clearly to you. But Lar, my communication was already clear starting with the title of my post.

The reason I posted was to provide evidence of the cluelessness of chess engines.

All the best,
Steve

One thing computers are particularly bad at, and are likely to remain so, is finding moves that make sense to humans in positions where there are many sequences of moves that lead to the same result, e.g. drawn endgames where they have an "advantage". Another (frustrating) example is having a computer solve a tactical test position and the PV shows the opponent making a reply to the key move that leaves him hopelessly lost because it sees the move a human would make is even worse. If you give it a won endgame, there is a good chance that, even though it is clueless, it will find a winning line in time.

[Jim Walker]

All I can say Eelco is that Rumsfeld deserved the award.
Jim

[Robert Flesher]

Here is another position that my 64-bit mp engines with 5-man tablebases claim is a White victory, yet I think it's surely a draw.

Who is claiming a White victory? If the engine at very large search depth doesn't manage to improve its initial evaluation (probably between +1 and +1.5), it is in fact claiming a draw.
The engine is telling you exactly what's going on: White has the advantage but apparently it's not enough to convert to a win.

Exactly, the only time a engine claims a win is when there is a mate anouncement.Every other evaluation is just a likely hood of winning/drawing.A stactic evaluation is a very useful clue that the position might be drawn and is quite a valuable tool which helps to study positions.
Of course not all static evals mean that draw is proven but at least you get the idea.

Hello Alex and Robert,

I understand what you wrote, but respectfully disagree.

When a chess engine's evaluation shows the symbol for a winning advantage along with a confirming numeric display, whether it remains static or not after subsequent iterations, the engine thus claims a win. After all, all endgames are either decisive or drawn. Thus, a static output of a win should never be read as the engine claiming a draw. Again, the position may indeed be a draw, but the engine is not claiming that.

I agree with you that a static evaluation can sometimes clue humans that the position is a draw in reality, but more often than not, the engine is correct in evaluating the position as victorious.

Similarly, I've seen engines evaluate endgames as equal when in reality one side can win.

That's why I've posted two simple endgame positions in the past two days where all the best engines' evaluations are seriously wrong. I hope that in so doing to encourage programmers to improve endgame evaluations/performances in their next engine generations.

I'd also like to see programmers offer an endgame version of their engines - an engine with adequate endgame knowledge to be used exclusively to analyze endgames.

All the best,
Steve

Steve, I agree with you, although, I just wanted to point out that Junior shows the least amount of advantage. But, cleary, the engine does not see the draw. The only engine I can remember that was good at some of these positions was an early version of Hiarcs (6) ? , I believe it had some opposite colored bishop knowledge included.

[jwes]

Here is another position that my 64-bit mp engines with 5-man tablebases claim is a White victory, yet I think it's surely a draw.

Who is claiming a White victory? If the engine at very large search depth doesn't manage to improve its initial evaluation (probably between +1 and +1.5), it is in fact claiming a draw.
The engine is telling you exactly what's going on: White has the advantage but apparently it's not enough to convert to a win.

Exactly, the only time a engine claims a win is when there is a mate anouncement.Every other evaluation is just a likely hood of winning/drawing.A stactic evaluation is a very useful clue that the position might be drawn and is quite a valuable tool which helps to study positions.
Of course not all static evals mean that draw is proven but at least you get the idea.

Hello Alex and Robert,

I understand what you wrote, but respectfully disagree.

When a chess engine's evaluation shows the symbol for a winning advantage along with a confirming numeric display, whether it remains static or not after subsequent iterations, the engine thus claims a win. After all, all endgames are either decisive or drawn. Thus, a static output of a win should never be read as the engine claiming a draw. Again, the position may indeed be a draw, but the engine is not claiming that.

I agree with you that a static evaluation can sometimes clue humans that the position is a draw in reality, but more often than not, the engine is correct in evaluating the position as victorious.

Similarly, I've seen engines evaluate endgames as equal when in reality one side can win.

That's why I've posted two simple endgame positions in the past two days where all the best engines' evaluations are seriously wrong. I hope that in so doing to encourage programmers to improve endgame evaluations/performances in their next engine generations.

I'd also like to see programmers offer an endgame version of their engines - an engine with adequate endgame knowledge to be used exclusively to analyze endgames.

All the best,
Steve

Steve, I agree with you, although, I just wanted to point out that Junior shows the least amount of advantage. But, cleary, the engine does not see the draw. The only engine I can remember that was good at some of these positions was an early version of Hiarcs (6) ? , I believe it had some opposite colored bishop knowledge included.

It is not that obvious that it is a draw. If you remove the minor pieces from the original position, it is a win for white. It might be interesting to try defending against the computer, giving it a long time for each move.