Calculation the playing strength of a move

[Jon12345]

I want to develop an algo that determines the playing strength of a given move. Let me explain...

You are in a middlegame, and choose the 3rd best move. Your positional evaluation goes from 0.2 to -0.75. What elo strength was that move?

Any ideas on how to solve this riddle?

[Steve Maughan]

It's an interesting and laudable goal to solve such a problem but it isn't easy.

My first thoughts would be to train a neural network using games of players of known rating and look at the attributes of the move made and the alternatives.

Steve

[Jon12345]

I was thinking you could use something like this: https://www.chessprogramming.org/Point_ ... n_Analysis

It gives the probability of a win based on material differences. Then you could look at the difference between the win probability before you made your move and after.

Or...

Assume you are playing Magnus. This allows you to use an "Elo per pawn" graph to calculate what that pawn loss of 0.95 (in my example) was worth.

Elo per pawn: https://wismuth.com/elo/calculator.html

[hgm]

I don't think you can judge this based on the score decrease. It all depends on how obvious it is that the move is sub-optimal. A move that results in the forced loss of a Queen through a combination so complex that most GM would not see it is a better move than 1. a3, which just loses a few centi-Pawn.

[Jon12345]

Essentially, are you are saying the playing strength of a move is not the real strength if it is hard to see, and therefore you should rank a complex combination that results in the forced loss of a Queen to be a less strong move than 1. a3?

[Uri Blass]

Essentially, are you are saying the playing strength of a move is not the real strength if it is hard to see, and therefore you should rank a complex combination that results in the forced loss of a Queen to be a less strong move than 1. a3?

If the target is to evaluate the playing strength of the player who played the move then losing a queen that is hard to see can be better than losing a pawn that is easy to see.

I do not know what is the target.

[Jon12345]

Edit: Let me rephrase my original reply: So if someone played a move with higher risk that would lose the Queen with accurate play, you should rank that moves strength as higher than if they played a different move without that risk?

2nd Edit: Please note, I said the strength of a given move, not a player.

[hgm]

Perhaps the misunderstanding is about what you mean by 'strength of a move' expressed in Elo. I imagined you meant that the move would be typically played by a player with a certain rating. Like for a given situation, GMs would typically prefer move A, but patzers would prefer another move, because it is without their grasp to see the merits of move A. My point was that this cannot be decided from the evaluation drop.

OTOH, if you mean "how much stronger does the player have to be to salvage the game after the move, compared to before the move" it is a different matter. You could roughly equate each centi-Pawn score loss with 1,3 Elo then. At least in an opening position. You need to be about 130 Elo stronger to have equal chances in a Pawn-odds game. This will increase as the game process, though; in many late end-games having an extra Pawn will be 100% decisive. A Patzer might not be able to beat Stockfish even when given Queen odds; he will have no problem to covert KQK, though.

There are still several problems with this, however. Engine scores are only heuristic. The game-theoretical value of a chess position is just win, draw or loss. All drawn positions are equivalent, and the only poor moves you can make are those that turn a win into a draw/loss, or those that turn a draw into a loss. All other moves are theoretically equivalent, no matter how much the evaluation drops when you make them. In practice this means that the depth of the analysis that provided the scores matters. When analyzed deep enough, the only remaining scores would be draws and mate scores. So there is no such thing as a score drop of a certain amount of centi-Pawn.

A wrong move that eventually leads to forced loss of a Queen in a complex way might initially look good or even best to the engine, until it reaches the search depth that gets the loss within the horizon. Then the score will drop dramatically. So which score are you going to take to determine the Elo of that move?

[xr_a_y]

A Patzer might not be able to beat Stockfish even when given Queen odds; he will have no problem to covert KQK, though.

This remark makes me try to play SF with Q odds on lichess, and it seems to me, SF really didn't play well, it proposed exchanges all game long...
I'm only 1600 on lichess, opened with f4, and played very fast here.

https://lichess.org/tD42MPgR/white#79

[pgn]
[Event "Casual Standard game"]
[Site "https://lichess.org/tD42MPgR"]
[Date "2021.07.11"]
[White "xr_a_y"]
[Black "lichess AI level 8"]
[Result "1-0"]
[UTCDate "2021.07.11"]
[UTCTime "12:25:13"]
[WhiteElo "1500"]
[BlackElo "?"]
[Variant "From Position"]
[TimeControl "-"]
[ECO "?"]
[Opening "?"]
[Termination "Normal"]
[FEN "rnb1kbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[SetUp "1"]
[Annotator "lichess.org"]

1. f4 e6 2. e3 Nf6 3. Nf3 b6 4. Bd3 Ba6 5. Bxa6 Nxa6 6. Nc3 Be7 7. b3 O-O 8. Bb2 Nc5 9. Qe2 Nd5 10. Nxd5 exd5 11. O-O-O Bf6 12. Bxf6 gxf6 13. g4 c6 14. h4 b5 15. g5 Ne4 16. Rdg1 f5 17. d3 Nd6 18. Ne5 Rfd8 19. h5 Kf8 20. g6 Ke7 21. gxh7 Rh8 22. Rg7 Rae8 23. Rhg1 Kf6 24. Nxd7+ Ke7 25. Ne5 Kd8 26. Ng6 fxg6 27. hxg6 a5 28. c4 Kc8 29. cxb5 c5 30. Qh5 c4 31. dxc4 Ref8 32. cxd5 Rd8 33. Re7 Kb8 34. g7 Rhg8 35. hxg8=Q Rxg8 36. Qh8 Rc8+ 37. Qxc8+ Nxc8 38. g8=Q a4 39. Qxc8+ Kxc8 40. Rg8# { White wins by checkmate. } 1-0
[/pgn]

[Ferdy]

I want to develop an algo that determines the playing strength of a given move. Let me explain...

You are in a middlegame, and choose the 3rd best move. Your positional evaluation goes from 0.2 to -0.75. What elo strength was that move?

Any ideas on how to solve this riddle?

Here is one approach.
1. Get the sigmoid or expected score of that 0.2.
2. Get the expected score of that -0.75.
3. Calculate the rating difference given the expected score using FIDE table of conversion section 8.1a for both scores.
4. Calculate the rating difference between the rating rating differences, that would be the Elo rating loss of the score of -0.75 by not finding the best 0.2.

Sample calculation:

Code: Select all

score1: +0.20, sigmoid1: +0.51250, rdiff: +8.69
score2: -0.75, sigmoid2: +0.45326, rdiff: -32.57
rating diff = -32.57 - (+8.69) = -41.26

If the initial rating of your opponent is say 2000, your perf rating will be 2000 - 32.57 or 1967.43. However if you find the 0.2 your perf rating will be 2000 + 8.69 or 2008.69.

See also the wiki on Elo.

Sigmoid or expected score:

Code: Select all

def sigmoid(score_cp):
    k = 1 / 400
    r = 1 / (1 + math.exp(-score_cp * k))
    return r