score from White's perspective, or engine's perspective?

[CThinker]

This is a classic error in presentation of implementation (exposing programming implementation to end-users).

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Presentation should be in non-technical terms and should be abstracted from programming implementations. Instead of numbers, it should be something like - ahead, behind, winning, losing, drawn, possibly winning, high probability of winning. The POV should also be clear (e.g., "The Rybka engine thinks that White is winning, being ahead in pieces").

Here is another example of error in presentation... The UI says "hash memory" or "hash size". Now, "hash" is a programming implementation. What if the engine uses AVL trees? Some engines have 4 different hash tables. The UI should just say something like "maximum memory usage", and leave it up to the engine to decide how that memory is used.

[xsadar]

If engines want to print mate scores, I propose they do it using scores in the 1,000,000 range, as follows:

1,000,001 = mate in 1 (1 ply)
1,000,002 = mate in 2 (3 ply)
...

-1,000,000 = checkmated
-1,000,001 = mated in 1 (2 ply)
...

The GUI can display these scores as M1, M2, ... and -M0, -M1, ..., respectively, if it wants.

This convention has the advantage that GUIs not implementing it would not choke on the engine output, as it is still a normal integer. And what they display in that case would still be easy to interpret by the user.

The one disadvantage I see with this, is that Arena, and perhaps some other GUIs, already have the capability of detecting some mate scores (like 32767 = 0x7fff as being mate in 1 ply). However, I don't think the current versions would detect the 1,000,000 range scores. Of course, once it was standardized, I would expect the GUIs to detect those scores in their next versions, but those scores would break the functionality of existing versions. Just something to consider when making the decision.

Also, something that's really a non-issue, but scores that move closer to zero as they move closer to mate just bother me a little, even though they are a little more human readable.

[AndrewShort]

just fyi, my own engine uses this for mate scores (from point of view of the engine):

999,999 means engine has mate in 1
999,997 means engine has mate in 2
etc.

-999,998 means engine can be mated in 1
-999,996 means engine can be mated in 2
etc.

obviously, the distance from mate is the ply where the mate would be, and the +/- is just because it's from the point of view of the engine.

[AndrewShort]

just fyi, my own engine uses this for mate scores (from point of view of the engine):

999,999 means engine has mate in 1
999,997 means engine has mate in 2
etc.

-999,998 means engine can be mated in 1
-999,996 means engine can be mated in 2
etc.

obviously, the distance from mate is the ply where the mate would be, and the +/- is just because it's from the point of view of the engine.

of course, that's just internal. I think ply distance is confusing to normal humans. Better that an offset of x really means mate in x, which ply distance doesn't give you...

[xsadar]

This is a classic error in presentation of implementation (exposing programming implementation to end-users).

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Presentation should be in non-technical terms and should be abstracted from programming implementations. Instead of numbers, it should be something like - ahead, behind, winning, losing, drawn, possibly winning, high probability of winning. The POV should also be clear (e.g., "The Rybka engine thinks that White is winning, being ahead in pieces").

Here is another example of error in presentation... The UI says "hash memory" or "hash size". Now, "hash" is a programming implementation. What if the engine uses AVL trees? Some engines have 4 different hash tables. The UI should just say something like "maximum memory usage", and leave it up to the engine to decide how that memory is used.

The score of +1.00 has a specific meaning -- ahead by one pawn's worth -- and is quite useful to the user. If you want a caption to accompany the score that's fine, but the actual score is still quite important.
While it's true that two engines will not necessarily agree completely on what it means to be ahead by a pawn, their scores are usually at least somewhat comparable. And one engines scoring of the current position and the previous position certainly are comparable, or if not, then the scoring is done wrong. Throwing out the scores is like throwing out the numbers on a ruler and just putting short, medium, long, really long, etc.

On the hash issue, however, I will have to agree with you. Who's idea was it to ask a non-programmer how much hash they want? If I wasn't a programmer, my first response might be something like: I wouldn't mind a plate full, but what does breakfast food have to do with chess? (I apologize to those of you outside the US who may not know what hash is in terms of food.)

[BubbaTough]

The score of +1.00 has a specific meaning -- ahead by one pawn's worth -- and is quite useful to the user. If you want a caption to accompany the score that's fine, but the actual score is still quite important.

Yeah, that always confused me. Assumably, higher score = higher chance to win. So +1 means the engine to move has a chance to win equal to being a pawn up. But a pawn up when? A pawn up at the start of the game I guess. Anyway, the whole issue is mostly theoretical since most engines are so material focussed (instead of chance to win focussed) in eval that +1 = up pawn, +3 = knight up, + 5 = up rook is close enough.

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Almost all the time, +3 is a win (in crafty). +5 is slightly more of a win...but drawing the distinction on which is a "won game" at +5 instead of +3 is a bit artificial. +3 is a won game...Crafty is probably just missing some knowledge in detecting certain drawn endgames (including the ACCA one you saw) and mistakenly thought it was winning when it was not. The same is possible at +5 too, just even less likely.

-Sam

[xsadar]

The score of +1.00 has a specific meaning -- ahead by one pawn's worth -- and is quite useful to the user. If you want a caption to accompany the score that's fine, but the actual score is still quite important.

Yeah, that always confused me. Assumably, higher score = higher chance to win. So +1 means the engine to move has a chance to win equal to being a pawn up. But a pawn up when? A pawn up at the start of the game I guess.

Ideally, I think it would mean the average value of a pawn, but I don't think it matters too much, since the scores are just estimates anyway. The important thing is that in related positions, the better position gets the better score, and that the sign on the score is correct. Of course chance to win (as ratio or percentage) would be more meaningful, but that's much more difficult to estimate (at least for me).

Anyway, the whole issue is mostly theoretical since most engines are so material focussed (instead of chance to win focussed) in eval that +1 = up pawn, +3 = knight up, + 5 = up rook is close enough.

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Almost all the time, +3 is a win (in crafty). +5 is slightly more of a win...but drawing the distinction on which is a "won game" at +5 instead of +3 is a bit artificial. +3 is a won game...Crafty is probably just missing some knowledge in detecting certain drawn endgames (including the ACCA one you saw) and mistakenly thought it was winning when it was not. The same is possible at +5 too, just even less likely.

-Sam

[Dann Corbit]

The score of +1.00 has a specific meaning -- ahead by one pawn's worth -- and is quite useful to the user. If you want a caption to accompany the score that's fine, but the actual score is still quite important.

Yeah, that always confused me. Assumably, higher score = higher chance to win. So +1 means the engine to move has a chance to win equal to being a pawn up. But a pawn up when? A pawn up at the start of the game I guess. Anyway, the whole issue is mostly theoretical since most engines are so material focussed (instead of chance to win focussed) in eval that +1 = up pawn, +3 = knight up, + 5 = up rook is close enough.

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Almost all the time, +3 is a win (in crafty). +5 is slightly more of a win...but drawing the distinction on which is a "won game" at +5 instead of +3 is a bit artificial. +3 is a won game...Crafty is probably just missing some knowledge in detecting certain drawn endgames (including the ACCA one you saw) and mistakenly thought it was winning when it was not. The same is possible at +5 too, just even less likely.

-Sam

I think in the case of Crafty, the score reported is in centipawns.
If crafty has any sort of possibilty to improve the score (e.g. imagine if the opponent does not have endgame tablebases or does not use them properly) then Crafty will try to bluff out a win.

Imagine (for instance) if Crafty as white had KRRkq in a tablebase drawn position. Crafty may play on and hope that the opponent blunders. Crafty has accurate tablebase access and as such can never decrease the performance of the position, and the opponent may make an error.

I think it would be nice for both programs to report that they have tablebase access, and so besides "offer draw" there could be a "offer tablebase draw" that says, "Not only is the position drawn, but I examined the tablebase files and see that it is a draw" in which case there is really no point in playing on.

[Dann Corbit]

This is a classic error in presentation of implementation (exposing programming implementation to end-users).

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Presentation should be in non-technical terms and should be abstracted from programming implementations. Instead of numbers, it should be something like - ahead, behind, winning, losing, drawn, possibly winning, high probability of winning. The POV should also be clear (e.g., "The Rybka engine thinks that White is winning, being ahead in pieces").

I guess that most of the time the engine will have a hard time telling us the real meaning of the score. If we just count wood, then all engines will tell us something stupid (e.g. switch your engine's evaluation function to pure wood counting and it will promptly lose every game against very low level opposition). The real reason for evaluation is going to be a complicated mix of evaluation terms and search terms. I think it would be an awful tangle if it were to try to tell us this stuff on every iteration. However, I think it would be very useful to have a blurb of human readable explanation at the point that the move is actually made. ChessMaster 9 has this feature "It's called something like 'human readable analysis'"

Here is another example of error in presentation... The UI says "hash memory" or "hash size". Now, "hash" is a programming implementation. What if the engine uses AVL trees? Some engines have 4 different hash tables. The UI should just say something like "maximum memory usage", and leave it up to the engine to decide how that memory is used.

[bob]

This is a classic error in presentation of implementation (exposing programming implementation to end-users).

Even if we agree on the +/-, what exactly does +1.0 mean? In the last ACCA games, Crafty outputs +3 on a drawn game. Bob comments that this does not imply a win. As I quote Bob, "+5 is winning in Crafty speak".

Not just "crafty speak" but "realistic speak". I can't think of a single program that I have not seen report an eval of more than +5 and then draw or even lose the game. And yes, this includes Rybka as well. +3 can be overwhelmingly winning, or it can simply be a piece ahead in a KRN + other pieces against a KR + other pieces so that the knight is not enough to win... Fortresses come to mind on drawing +5 endgames as well.

Presentation should be in non-technical terms and should be abstracted from programming implementations. Instead of numbers, it should be something like - ahead, behind, winning, losing, drawn, possibly winning, high probability of winning. The POV should also be clear (e.g., "The Rybka engine thinks that White is winning, being ahead in pieces").

Here is another example of error in presentation... The UI says "hash memory" or "hash size". Now, "hash" is a programming implementation. What if the engine uses AVL trees? Some engines have 4 different hash tables. The UI should just say something like "maximum memory usage", and leave it up to the engine to decide how that memory is used.