TalkChess.com

[Rémi Coulom]

Thanks Edmund for your interesting data.

I certainly won't have time to modify bayeselo in the near or even not-so-near future. But bayeselo lets you set the DrawElo and EloAdvantage. You can also find the maximum-likelihood values for these parameters based on your data. That should be good enough when testing programs that don't have a huge difference in playing strength.

Another important question is the model for the draw distribution. I tried to compare a few different models some time ago, but could not reach a clear conclusion.

Comparing frequencies with what the model predicts is interesting, but it should be considered carefully. If the frequency does not match the model, then it is a sign that the model is bad. But if it does match the model, it does not mean that the model is good, because the ratings were computed with the model in the first place.

The only really proper way to compare models is to measure the quality of predictions they make. The main problem is that this approach requires huge amounts of data in order to be able to make statistically significant comparisons.

Rémi