The flow of Komodo eval

[Laskos]

With the help of a tool written by Ferdinand Mosca, I plotted the evolution of eval of Komodo eval starting with positions at certain move number, tracked for 10 moves (not more, because they disperse too much for longer series of moves and the average becomes a bit meaningless). The analyzed games were 3000 Komodo self-games at 240''+2.4''. To read this plot: say a worm-like line begins at move 20 with an eval of 1.17. It is expected the eval reaches a value of 1.56 at move 30, and all intermediate values are plotted too (well, the lines are worm-like because of the noise). Eval in Chess as a dynamic system has three fixed points attractors: Win, Draw and Loss.We empirically know that say an eval of above +2.00 will go towards Win pretty much independently of move number, and the eval of +0.10 at move 70 is probably a Draw. Using game statistics function by eval (another script by Ferdinand), I also colored the background in two different shades: darker one - game tends to a Draw, lighter one - game tends to a Win.

The second plot is the fitted function - vector field. The fit is not hard, as the vector field of worm-like lines is optically close to {x, y^2} with a small adjustment to account for a tendency for Draws at small evals. The arrows fit my worms pretty closely. Here too, darker region is a flow towards mostly draws, lighter one - towards mostly wins.

[Dann Corbit]

If the score is above 70 cp on the 10th move, you are trapped in the vector field of doom.

[nimh]

Thanks, it is very interesting and informative!

I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.

So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?

Could you give the formula, so I can check it myself on my data?

[Laskos]

Thanks, it is very interesting and informative!

I wonder if translating centipawn evals into win-draw-loss transitions is actually more trustworthy than into expected scores via a logistic function.

So, when I see a player has made a move whose eval drops from 0.96 to -1.02 on move 15th, I just enter 2 for two transitions, as a won game turned into a lost game?

Could you give the formula, so I can check it myself on my data?

Formula for vector field lines? The win/draw boundary is close to be one of them. To solve y[x] for the vector field {x, y^2} we have the simple differential equation

y'[x] * x = y[x]^2
y[move(0)] = eval(0)

getting

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

For example:
move(0) = 20 (we start at move 20).
eval(0) = 0.7 (we start with an eval of 0.7)

We get from the above formula eval at move 30 (or y[30]) to be equal to 0.977. But keep in mind that this is a pretty rough approximation, as I didn't keep in this solution the small additional term in the vector field needed to fit it better to my "worms", differential equation doesn't solve in closed form including it.

[nimh]

Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.

[Laskos]

Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.

But that's easy to translate:

having

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

and picking the initial point on the boundary (say 30, 0.75), we get

y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number

And the needed formula is

0.5 + 0.5 * theta(y[x] - y_b[x])

[JJJ]

How strong are these value against his rival and slighty better Stockfish ?

[Laskos]

Thanks, but what I had in mind is a formula that returns 0.5 when below, and 1.0 when above the boundary.

But that's easy to translate:

having

y[x] = eval(0) / (1 + eval(0)*log[move(0)] - eval(0)*log[x])

and picking the initial point on the boundary (say 30, 0.75), we get

y_b[x] on boundary = 0.75 / (1+0.75*log[30]-0.75*log[x])
x is the variable move number

And the needed formula is

0.5 + 0.5 * theta(y[x] - y_b[x])

I managed a good fit solvable in closed form. Here is how the fit to my worms looks like, only the small eval values need a correction, but introducing it makes the differential equation unsolvable (well, numerically it's solvable for given boundary conditions), at least by Mathematica.



The vector field is:
{x, 0.1*x^(1/2)*y^2}

The differential equation for the flow lines is:
y'[x] = 0.1*x^(-1/2)*y[x]^2
y[move(0)] = eval(0)

The solution to this (better) fit is even simpler than before:
y[x] = 5*eval(0)/{5 - eval0*sqrt[x] + eval0*sqrt[move(0)]}

From this family of flow lines, I picked the a particular boundary condition for the Win/Draw border line, and the border equation is:

y_b[x] = 5*0.75/(5 + 0.75*sqrt(30) - 0.75*sqrt(x))
x is the variable move number.

You probably need the same:
f[x] = 0.5 + 0.5 * theta(y[x] - y_b[x])

Keep in mind that the border is soft, there is no phase-transition between Wins and Draw, the border is simply showing where the probability of Draw is equal to probability of Win.

[Laskos]

How strong are these value against his rival and slighty better Stockfish ?

If SF is sensibly better, you probably need to shift a bit the whole graph up on vertical axis, so what was before (self-games) "eval" takes say value "eval + 0.1" against SF.

[nimh]

I've never been interested in maths, I'm weak at it.