Calculation the playing strength of a move

[Ferdy]

That 3000 is only an example, it does not apply to players of around 1500 . The reference engine rating has to be determined wisely depending on the players rating.

Here is what I had experimented before. Player ratings as seen by the engine at fixed depth 16. The rating is based on moves's evaluation by the engine. If the player move is the same as the engine's best move at depth 16, a player will get the 2800.

Code: Select all

minimum move: 8
Skilling Open 2020 rating list according to Stockfish 13 @depth 16 set at 2800
 rank                    name  games  points  rating
    1 Vachier-Lagrave, Maxime     15     8.0    2780
    2             Ding, Liren     15     7.5    2778
    3        Nakamura, Hikaru     15     9.0    2772
    4              So, Wesley     15     8.5    2772
    5          Aronian, Levon     15     8.5    2771
    6       Radjabov, Teimour     15     8.0    2771
    7         Carlsen, Magnus     15     9.0    2770
    8 Vidit, Santosh Gujrathi     15     6.5    2770
    9             Giri, Anish     15     8.0    2769
   10     Duda, Jan-Krzysztof     15     4.5    2767
   11     Nepomniachtchi, Ian     15     8.5    2766
   12          Le, Quang Liem     15     8.0    2765
   13          Svidler, Peter     15     6.0    2760
   14       Firouzja, Alireza     15     8.0    2753
   15        Karjakin, Sergey     15     5.5    2752
   16   Anton Guijarro, David     15     6.5    2739

[hgm]

Isn't that the same as saying that the Elo you have to assign to the move has to be determined from the rating of the player playing it, in an indirect way?

[Jon12345]

Your proposed method seems to assume that the result of a game is determined by a single sub-optimal move.

My view is that this proposed method doesn't assume the result of the game is determined by a single sub-optimal move. It only looks at probabilities. So, if you make a sub-optimal move, the probability of you winning goes down a little. Each sub-optimal move accumulates your losing chances, given best moves by the opponent.

The rating of an individual move is not an indicator of who wins the game, since it is only the aggregation of all moves that gives a result. It just has predictive value.

You want to express the quality of the move as a number. Not just as the verdict "this move is not the best".

Yes I agree. Stockfish currently gives the best estimate of a position that we have, better than any human. To me, that is an objective evaluation of a position. I think part of this discussion revolves around the interpretation of what "objective" means. Objective doesn't necessarily mean 100% accurate, with all moves calculated to the end of the game, although for some that is the definition.

But let us assume that the Stockfish evaluation was way off in magnitude, just for kicks. You could still develop a predictive model based on creating a sigmoid distribution (if that is the right terminology) from those Stockfish games. Then the magnitude becomes irrelevant, because the strength of all the moves are then based on Stockfish's evaluation function, as is the sigmoid. They are in sync.

In either case, you can develop that predictive model, although having to create your own sigmoid is beyond my limited abilities. Instead, I would like to rely on Stockfish having an objective evaluation of a position. For an objective evaluation, I don't mean using 1/100th of a second processing time. As we all know, the longer the processing time, the stronger the objective evaluation.

[Jon12345]

That 3000 is only an example, it does not apply to players of around 1500 . The reference engine rating has to be determined wisely depending on the players rating.

Here is what I had experimented before. Player ratings as seen by the engine at fixed depth 16. The rating is based on moves's evaluation by the engine. If the player move is the same as the engine's best move at depth 16, a player will get the 2800.

That is very interesting in what you have done there. What is the rational about capping the move rating at 2800? I was going to do the same, but capping at Magnus's rating. Is it anything to do with Stockfish's rating at 16 ply, or just so that the estimated move strengths of the players doesn't look bonkers?

I see that Magnus's 7th position would suggest that he players riskier chess, although that seems counter-intuitive. Perhaps he keeps risking Harry as he sends him up the board.

Edit: Just to be clear, I am assuming the rating score for each player is based on an average of all that players moves in that game. Or in other words, it is the average playing strength of all their moves, yes?

[hgm]

My view is that this proposed method doesn't assume the result of the game is determined by a single sub-optimal move. It only looks at probabilities. So, if you make a sub-optimal move, the probability of you winning goes down a little. Each sub-optimal move accumulates your losing chances, given best moves by the opponent.

If your score goes down 0.95 in a single move, your winning probability doesn't go down a little; it goes down a lot. You just gave away a full Pawn.

If you give away a full Pawn on every move, how much weaker than your opponent do you think you are? 41 Elo?

[Jon12345]

If your score goes down 0.95 in a single move, your winning probability doesn't go down a little; it goes down a lot. You just gave away a full Pawn.

Not necessarily. It depends on what part of the sigmoid you are on. If it is near either end of it, a 0.95 loss has very little impact on winning probability. But you and I both know this has nothing to do with the point I was making.

If you give away a full Pawn on every move, how much weaker than your opponent do you think you are? 41 Elo?

How would you judge?

[hgm]

If your score goes down 0.95 in a single move, your winning probability doesn't go down a little; it goes down a lot. You just gave away a full Pawn.

Not necessarily. It depends on what part of the sigmoid you are on. If it is near either end of it, a 0.95 loss has very little impact on winning probability. But you and I both know this has nothing to do with the point I was making.

Well, since the 0.95 score drop refers to the +0.20 to -0.75 in your initial posting, you know very well which part of the sigmoid we are on.

My guess is that you will invite a lot of ridicule when you will be presenting a move as a 2860-Elo move because in an analysis with a 3000-Elo engine it 'only' blunders a Pawn away in an approximately equal position. But hey, if you want to try that, who am I to argue against it?

[Jon12345]

Well, since the 0.95 score drop refers to the +0.20 to -0.75 in your initial posting, you know very well which part of the sigmoid we are on.

My comment you replied to was about the accumulation of losing chances, not one move. Have another read of it.

My guess is that you will invite a lot of ridicule when you will be presenting a move as a 2860-Elo move because in an analysis with a 3000-Elo engine it 'only' blunders a Pawn away in an approximately equal position. But hey, if you want to try that, who am I to argue against it?

I think using formulas for how to calculate your elo change is different from calculating the strength of a move. They seem similar, but the difference is subtle. For example, if a beginner played Magnus and lost, they would only lose 500 elo on that one game. But they are clearly not 2300 strength.

I think I have a solution though.

On average, a player of a certain elo will play x units below the average engine suggestion. Find what that average is. Then, match your actual move to what the previous analysis suggested to give an estimate of move strength.

Perhaps one way you can do this is by working out the average length of a game, then use the formula shown under "Mathematical details" on this link:

https://en.wikipedia.org/wiki/Elo_rating_system

It gives you elo prediction if you feed in the engine rating and expected score.

Next, get the evaluation of the position at what the average game length is. Divide by the number of moves, and you have the average loss per move for a player of that strength.

Use the sigmoid to give a prediction of the result on that average move length. Then use the Mathematical details formula to predict the elo

So if average game length was 24, and resultant position is 5 pawns behind on average, you are losing 0.2 pawns per move on average. The sigmoid predicts the probability of the game result. Feed that into the mathematical details formula to predict the elo.

[hgm]

My comment you replied to was about the accumulation of losing chances, not one move. Have another read of it.

Your comment that I replied to was presented as a response to my (quoted) remark on the score loss in a single move. Perhaps you should have another read of it?

[Ferdy]

Isn't that the same as saying that the Elo you have to assign to the move has to be determined from the rating of the player playing it, in an indirect way?

In the experiment, initially we don't know the rating of players. We will only know its strength depending on the opponents moves (good or bad or equal) and how the player would reply to that move (good or bad or equal), the engine is the judge.

We instead assign to them a sort of a temporary rating of 2800 in this experiment. If player move is the same as that of SF or different moves but have equal evaluations, it still gets its rating of 2800, however if the move is suboptimal according to the engine, the 2800 is subtracted by the elo loss. There is also a possibility that the player move is stronger than the engine move and in this case the rating of that move is more than 2800.