Statistical interpretation of search and eval scores.

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
jwes
Posts: 773
Joined: Sat Jul 01, 2006 5:11 am

Statistical interpretation of search and eval scores.

Post by jwes » Sat Nov 18, 2017 11:19 pm

My math and stats are pretty rusty, but I seem to recall a good estimate would have an unbiased mean and normally distributed errors. I don't see how scores in centipawns can have these properties, as the true value is win, draw or loss. Can someone help me understand what form of scores would have these properties?

Robert Pope
Posts: 397
Joined: Sat Mar 25, 2006 7:27 pm

Re: Statistical interpretation of search and eval scores.

Post by Robert Pope » Sun Nov 19, 2017 3:49 am

My $0.02;

1. The score isn't an estimate of the game theoretic value (win/draw/loss). We are estimating the expected outcome over the universe of games that would be played that contain that position.

2. You are quite frequently doing some sort of transform before you create your estimator. You hope the errors of the transformed estimators are normally distributed, but that doesn't mean the untransformed errors would be.

jdart
Posts: 3507
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Statistical interpretation of search and eval scores.

Post by jdart » Sun Nov 19, 2017 4:38 am

The usual way to turn scores into a outcome predictor is to use a sigmoid function to transform the values -MATE .. MATE into a 0..1 range (0 being 100% probability of loss, and 1 being 100% probability of win). A scaling factor may have to be applied in the function to get the probabilities correct. But this seems to be empirically sound, at least. It is also a standard model in the machine learning community (Logistic Regression).

Since there are three possible game outcomes though, for machine learning Ordinal Logistic Regression is possibly a sounder approach.

--Jon

DustyMonkey
Posts: 25
Joined: Wed Feb 19, 2014 9:11 pm

Re: Statistical interpretation of search and eval scores.

Post by DustyMonkey » Mon Nov 20, 2017 4:38 pm

A static eval score is used to determine an ordering for all positions. While in practice we can loosely convert this value into something differently "meaningful" such as "centipawn advantage" or "win probability", that isnt its actual purpose.

Post Reply