engine evaluation and chess informant symbols

[Don]

...one more think on the subject of translating engine evals into chess informant notations: what would be the criteria to assign ? and ?? signs?

The first idea would be to look for moves that cause of an evaluation drop against the player that just moved; for example with Black to move the engine eval is -1, after Black's move the enging eval goes to -0.3: that would be an evaluation drop of 0.7 against Black.

Any other suggestion than this for assigning ? and ?? symbols?

If that model is used, I'm also wondering what would be sensible value drop thresholds for ? and ??.

This could possibly be added to the quantitative analysis, correlating engine drops and marks assigned by trustworthy annotators.

Another approach would be to link those values to the thresholds for the -+, -/+, =/+ and so on, so that a ? would more or less correspond to to make the evaluation score worse by one step and a ?? would make the evaluation score worse by two steps or more... orsomething similar...

I have actually done some work on this. For ? or ! moves you must be less sensitive when one side is clearly ahead and more sensitive when one side is clearly behind or it just looks stupid. If you are 2 pieces up and fail to make a slight improvement to your position, it is not considered a blunder. But the same score drop might be considered a blunder if you are a bit better and the move makes you worse. I don't have any values to suggest, this is a bit of a black art but you make an arbitrary decision and then get feedback from people using it and adjust accordingly.

The problem with "!" moves is that there is no such thing, it is purely subjective. The best definition I could suggest is that if the score and move suddenly change at a high depth (to the positive) - give it an exclamation. That combination implies that the move was difficult to find and could be in some sense considered brilliant as it take a "high depth" to see it. Probably you would also want to make this a little more sensitive to scores near zero but I don't' think it matters as much as the question mark case. If you are a queen up and then suddenly discover checkmate (with a different move) at a high depth then it's probably not particular appropriate to heap praise on the move. When it saves the game or brings the position from bad to good it is especially praiseworthy.

For the sake of your tool you would probably would simply set some threshold of nodes. For example after N nodes have transpired and the javascript program suddenly changes moves with some rise in score, give it an exclamation. Perhaps you could "take it back" if later it proves to be bogus by changing again.

[casaschi]

...one more think on the subject of translating engine evals into chess informant notations: what would be the criteria to assign ? and ?? signs?

The first idea would be to look for moves that cause of an evaluation drop against the player that just moved; for example with Black to move the engine eval is -1, after Black's move the enging eval goes to -0.3: that would be an evaluation drop of 0.7 against Black.

Any other suggestion than this for assigning ? and ?? symbols?

If that model is used, I'm also wondering what would be sensible value drop thresholds for ? and ??.

This could possibly be added to the quantitative analysis, correlating engine drops and marks assigned by trustworthy annotators.

Another approach would be to link those values to the thresholds for the -+, -/+, =/+ and so on, so that a ? would more or less correspond to to make the evaluation score worse by one step and a ?? would make the evaluation score worse by two steps or more... orsomething similar...

I have actually done some work on this. For ? or ! moves you must be less sensitive when one side is clearly ahead and more sensitive when one side is clearly behind or it just looks stupid. If you are 2 pieces up and fail to make a slight improvement to your position, it is not considered a blunder. But the same score drop might be considered a blunder if you are a bit better and the move makes you worse. I don't have any values to suggest, this is a bit of a black art but you make an arbitrary decision and then get feedback from people using it and adjust accordingly.

The problem with "!" moves is that there is no such thing, it is purely subjective. The best definition I could suggest is that if the score and move suddenly change at a high depth (to the positive) - give it an exclamation. That combination implies that the move was difficult to find and could be in some sense considered brilliant as it take a "high depth" to see it. Probably you would also want to make this a little more sensitive to scores near zero but I don't' think it matters as much as the question mark case. If you are a queen up and then suddenly discover checkmate (with a different move) at a high depth then it's probably not particular appropriate to heap praise on the move. When it saves the game or brings the position from bad to good it is especially praiseworthy.

For the sake of your tool you would probably would simply set some threshold of nodes. For example after N nodes have transpired and the javascript program suddenly changes moves with some rise in score, give it an exclamation. Perhaps you could "take it back" if later it proves to be bogus by changing again.

Thanks for the feedback.

The idea of linking ? and ?? to the annotation symbols is indeed to avoid assigning a ? just because you are two pieces up and failed to grap a pawn. For now I'm experimenting something like this: if the annotation symbol changes of two steps, like from = to +/- after a Black move, then I mark as "?". If it changes three steps, like from -+ to = after a Black move, I mark as "??"

I'm not bothered by "!" and "!!", I follow Huebner idea that those dont make any sense other that to show the excitement of the annotator: you cannot improve the eval of a position, you can only make it worse. If the eval improves for you after your move then it means than either the previous or the current eval is wrong and it's not trivial to tell which...

[Don]

...one more think on the subject of translating engine evals into chess informant notations: what would be the criteria to assign ? and ?? signs?

The first idea would be to look for moves that cause of an evaluation drop against the player that just moved; for example with Black to move the engine eval is -1, after Black's move the enging eval goes to -0.3: that would be an evaluation drop of 0.7 against Black.

Any other suggestion than this for assigning ? and ?? symbols?

If that model is used, I'm also wondering what would be sensible value drop thresholds for ? and ??.

This could possibly be added to the quantitative analysis, correlating engine drops and marks assigned by trustworthy annotators.

Another approach would be to link those values to the thresholds for the -+, -/+, =/+ and so on, so that a ? would more or less correspond to to make the evaluation score worse by one step and a ?? would make the evaluation score worse by two steps or more... orsomething similar...

I have actually done some work on this. For ? or ! moves you must be less sensitive when one side is clearly ahead and more sensitive when one side is clearly behind or it just looks stupid. If you are 2 pieces up and fail to make a slight improvement to your position, it is not considered a blunder. But the same score drop might be considered a blunder if you are a bit better and the move makes you worse. I don't have any values to suggest, this is a bit of a black art but you make an arbitrary decision and then get feedback from people using it and adjust accordingly.

The problem with "!" moves is that there is no such thing, it is purely subjective. The best definition I could suggest is that if the score and move suddenly change at a high depth (to the positive) - give it an exclamation. That combination implies that the move was difficult to find and could be in some sense considered brilliant as it take a "high depth" to see it. Probably you would also want to make this a little more sensitive to scores near zero but I don't' think it matters as much as the question mark case. If you are a queen up and then suddenly discover checkmate (with a different move) at a high depth then it's probably not particular appropriate to heap praise on the move. When it saves the game or brings the position from bad to good it is especially praiseworthy.

For the sake of your tool you would probably would simply set some threshold of nodes. For example after N nodes have transpired and the javascript program suddenly changes moves with some rise in score, give it an exclamation. Perhaps you could "take it back" if later it proves to be bogus by changing again.

Thanks for the feedback.

The idea of linking ? and ?? to the annotation symbols is indeed to avoid assigning a ? just because you are two pieces up and failed to grap a pawn. For now I'm experimenting something like this: if the annotation symbol changes of two steps, like from = to +/- after a Black move, then I mark as "?". If it changes three steps, like from -+ to = after a Black move, I mark as "??"

I'm not bothered by "!" and "!!", I follow Huebner idea that those dont make any sense other that to show the excitement of the annotator: you cannot improve the eval of a position, you can only make it worse. If the eval improves for you after your move then it means than either the previous or the current eval is wrong and it's not trivial to tell which...

Yes, that is exactly what ! and !! is for. Like I say it's purely subjective. However the annotator is the javascript engine and it could indeed communicate it's excitement with those. The rule I proposed would be something you could do later if the mood strikes but certainly low on the list of priorities. I think the question mark moves are much more valuable for the players looking the tool for a clue about what is going on.