statistic magicians help required

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Posts: 774
Joined: Sun May 23, 2010 11:32 am

statistic magicians help required

Post by elcabesa » Wed Jul 11, 2018 8:55 pm

I'd like to make some assumption and I need some good knowledge of statistics that I don't have.

Those are my assumptions, please correct me where I'm wrong.

cutechess cli reports elo and a uncertainity window.

Score of Vajolet dev vs Vajolet base: 2425 - 2290 - 5586 [0.507] 10301
Elo difference: 4.55 +/- 4.53

here the uncertainity is +/- 4.53, is this a confidence interval of 95% in sense of gaussian variable?
in this case sigma is 4,53 /2 = 2,265 and variance is 2,265^2 = 5,130225

assuming that I do 2 tournaments and assuming the result statistically indipendent ( problably this is not true at all....) and that Elo results is a gaussian probablity variable we have:

Elo_gain = elo1 + elo2
sigma_tot^2 = sigma1^2 + sigma2^2

so I have combined result of elo1+elo2 +/- 2*sqrt(sigma_tot^2)

ok.. magician please let me know if I have made some orrible errors with variance, signa, statistical independence and so on.... thank you all

Posts: 910
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: statistic magicians help required

Post by AlvaroBegue » Wed Jul 11, 2018 9:30 pm

First of all, I think they report 1.96 standard deviations, not 2. But that's small potatoes.

You haven't been precise enough in describing the two-tournaments scenario. If you do two tournaments between the same two engines and you obtain Elo estimates with means Elo1, Elo2 and corresponding standard deviations sigma1, sigma2, I think the combined distribution is something like

mean = (sigma2^2 * Elo1 + sigma1^1 * Elo2) / (sigma1^2 + sigma2^2)
standard deviation = sqrt((sigma1^2 * sigma2^2) / (sigma1^2 + sigma2^2))

If you have a tournament A-vs-B and a tournament B-vs-C, then you can deduce the performance of A-vs-C with essentially the formulas you have
Elo(A-vs-C) = Elo(A-vs-B)+Elo(B-vs-C)
sigma(A-vs-C) = sqrt(sigma(A-vs-B)^2 + sigma(B-vs-C)^2)

Posts: 774
Joined: Sun May 23, 2010 11:32 am

Re: statistic magicians help required

Post by elcabesa » Thu Jul 12, 2018 4:35 am

thank you for your reply.
The scenario is the latter, A vs B and then B vs C

Post Reply