Simple little online ELO calculator

[gladius]

For 1v1 matches, it's nice to have a really simple way to get ELO. There are existing javascript ELO pages out there, but they don't offer LOS, or error bounds. BayesELO is awesome, but requires a bit more setup.

http://forwardcoding.com/projects/ajaxchess/rating.html is a completely barebones heads up match ELO calculator in javascript.

Let me know if you spot any problems.

Example output:
ELO: 6.57 +- 99%: 43.98 95%: 33.32
LOS: 73.51%
Wins: 65 Losses: 57 Draws: 301

[Ajedrecista]

Hello Gary:

For 1v1 matches, it's nice to have a really simple way to get ELO. There are existing javascript ELO pages out there, but they don't offer LOS, or error bounds. BayesELO is awesome, but requires a bit more setup.

http://forwardcoding.com/projects/ajaxchess/rating.html is a completely barebones heads up match ELO calculator in javascript.

Let me know if you spot any problems.

Example output:
ELO: 6.57 +- 99%: 43.98 95%: 33.32
LOS: 73.51%
Wins: 65 Losses: 57 Draws: 301

This is an excellent idea! However, I can spot some failures:

In your example of +65 -57 =301 I get the following results with my own Fortran tool:

Code: Select all

LOS_and_Elo_uncertainties_calculator, ® 2012.

----------------------------------------------------------------
Calculation of Elo uncertainties in a match between two engines:
----------------------------------------------------------------

(The input and output data is referred to the first engine).

Please write down non-negative integers.

Maximum number of games supported: 2147483647.

Write down the number of wins (up to 1825361100):

65

Write down the number of loses (up to 1825361100):

57

Write down the number of draws (up to 2147483525):

301

 Write down the confidence level (in percentage) between 65% and 99.9% (it will be rounded up to 0.01%):

95

Write down the clock rate of the CPU (in GHz), only for timing the elapsed time of the calculations:

3

---------------------------------------
Elo interval for 95.00 % confidence:

Elo rating difference:      6.57 Elo

Lower rating difference:  -11.20 Elo
Upper rating difference:   24.38 Elo

Lower bound uncertainty:  -17.77 Elo
Upper bound uncertainty:   17.81 Elo
Average error:        +/-  17.79 Elo

K = (average error)*[sqrt(n)] =  365.93

Elo interval: ] -11.20,   24.38[
---------------------------------------

Number of games of the match:       423
Score: 50.95 %
Elo rating difference:    6.57 Elo
Draw ratio: 71.16 %

*********************************************************
Standard deviation:  2.5573 % of the points of the match.
*********************************************************

 Error bars were calculated with two-sided tests; values are rounded up to 0.01 Elo, or 0.01 in the case of K.

-------------------------------------------------------------------
Calculation of likelihood of superiority (LOS) in a one-sided test:
-------------------------------------------------------------------

LOS (taking into account draws) is always calculated, if possible.

LOS &#40;not taking into account draws&#41; is only calculated if wins + loses < 16001.

LOS &#40;average value&#41; is calculated only when LOS &#40;not taking into account draws&#41; is calculated.
______________________________________________

LOS&#58;  76.57 % &#40;taking into account draws&#41;.
LOS&#58;  76.46 % &#40;not taking into account draws&#41;.
LOS&#58;  76.51 % &#40;average value&#41;.
______________________________________________

These values of LOS are rounded up to 0.01%

End of the calculations. Approximated elapsed time&#58;   79 ms.

Thanks for using LOS_and_Elo_uncertainties_calculator. Press Enter to exit.

· I get a LOS value of more than 76%! I have two methods: my own (76.57%) and other (76.46%) that was proposed by Rémi Coulom here. As you can see, both methods (Rémi's and mine) give very similar results (around 0.11% for only 423 games, and that difference gets smaller with more games).

· The error bars are not the same as mine because I include the draw ratio in the formula of standard deviation while you not. It is not an error but simply different models. It can not be considered an error.

· LOS again: I just tried +57 -57 =301 (an even score) and the expected LOS value should be 50%, but I get 46.3% with your tool!

Code: Select all

ELO: 0.00  +- 99%: 44.29 95%: 33.57
LOS: 46.30%
Wins: 57 Losses: 57 Draws: 301

There is something wrong.

· Another LOS bug IMHO: there is not simmetry in your LOS calculation. If I introduce +57 -65 =301 in your tool, I would expect 100% - LOS if +65 -57 =301 has LOS.

Code: Select all

ELO: -6.57  +- 99%: 43.77 95%: 33.20
LOS: 20.95%
Wins: 57 Losses: 65 Draws: 301

But 100% - 73.51% = 26.49%, which is different than 20.95%.

------------

Just a suggestion: you could add the number of games (wins + draws + loses) in the output. Good luck with your bug hunt! It is indeed a very good idea having this tool online. Please ask if you have doubts regarding my bug report.

Regards from Spain.

Ajedrecista.

[gladius]

Thanks for the bug report! I will take a look at why the LOS numbers are off.

[lucasart]

For 1v1 matches, it's nice to have a really simple way to get ELO. There are existing javascript ELO pages out there, but they don't offer LOS, or error bounds. BayesELO is awesome, but requires a bit more setup.

http://forwardcoding.com/projects/ajaxchess/rating.html is a completely barebones heads up match ELO calculator in javascript.

Let me know if you spot any problems.

Example output:
ELO: 6.57 +- 99%: 43.98 95%: 33.32
LOS: 73.51%
Wins: 65 Losses: 57 Draws: 301

Using a (slightly simplified) BayesElo equation:

Code: Select all

L(x) = 1/[1+10^(x/400)]
(1) L(drawelo - elo) = P(win)
(2) L(elo + drawelo) = P(loss)

And substituting with the empirical values

Code: Select all

W = P_hat(win) = #win / #games
L = p_hat(loss) = #loss / #games

You can easly solve the equations (1) and (2). In particular

Code: Select all

L^-1(y) = 400.log10(1/y-1)

So, assuming I didn't make any calculation mistake:

Code: Select all

elo = 200.log10[W/L.(1-L)/(1-W)]
drawelo = 200.log10[(1-L)/L.(1-W)/W]

So I get

Code: Select all

elo = +13.3
drawelo = 310

The calculation of the LOS is trivial: just mean and stdev and gaussian inverse distribution distribution. I get:

Code: Select all

mu = E(Xi) = 50.9%
sigma = sqrt(V(Xi)/N) = 1.31%
LOS = Phi^-1[(mu-.5)/sigma] = 76.5%

[lucasart]

· I get a LOS value of more than 76%! I have two methods: my own (76.57%) and other (76.46%) that was proposed by Rémi Coulom

To reach 76.57%, I think you made a mistake too
Don't forget that the denominator is N-1, not N, in the empirical variance (otherwise the estimator is biaised). If you correct this, you should find 76.54%, like I did.

[Ajedrecista]

Hello:

@Lucas:

· I get a LOS value of more than 76%! I have two methods: my own (76.57%) and other (76.46%) that was proposed by Rémi Coulom

To reach 76.57%, I think you made a mistake too
Don't forget that the denominator is N-1, not N, in the empirical variance (otherwise the estimator is biaised). If you correct this, you should find 76.54%, like I did.

In fact I divide by n instead of (n - 1); I know the fact of biased and unbiased estimators, but bearing in mind that I also use an approximation of normal distribution that works very well for a high amount of games, it is the same for me including one approximation (the normal distribution) or two (the normal distribution and dividing by n). Of course it has an easy solution, but my point is giving reasonable LOS values: talking about 76%, I accept errors of 0.1% (like this case), but an error of 3% is too much. You can see that my aim is not very professional but at least I obtain reasonable values!

If I compare both standard deviations, they will only change in sqrt[(n - 1)/n] = sqrt(1 - 1/n) or its inverse. If n >> 1 then sqrt(1 - 1/n) ~ 1 - 0.5/n (or its inverse 1 + 0.5/n), which is the unity more less. But of course you are right. Good Statistics reminder in the General Forum!

------------

@Gary: I think you are trying to use WhoIsBest by Rémi Coulom. You are doing the following in your web:

Code: Select all

<!-- saved from url=&#40;0014&#41;about&#58;internet --> 
<html> 
  <head> 
    <title>ELO ratings</title> 
    <link rel="stylesheet" href="../../style.css" type="text/css">
    <script type="text/javascript" src="js/jquery-1.3.2.min.js"></script> 
    <script type="text/javascript" src="js/jquery-ui-1.7.2.custom.min.js"></script>
    <script type="text/javascript">
      function los&#40;n1, n0&#41; &#123;
        var pi = &#91;&#93;;
        for &#40;var i = 0; i <= n1 + 1; ++i&#41;
          pi&#91;i&#93; = 1.0;
        pi&#91;0&#93; = 0.0;
 
        for &#40;var j = n0 + 1; j >= 0; --j&#41;
          for &#40;var i = 0; i <= n1; ++i&#41;
            pi&#91;i + 1&#93; = &#40;pi&#91;i + 1&#93; + pi&#91;i&#93;) * 0.5;
 
        return pi&#91;n1 + 1&#93; * 100.0;
      &#125;
 
      function elo&#40;winRatio&#41; &#123;
        return 400.0 * &#40;Math.log&#40;winRatio / &#40;1 - winRatio&#41;) / Math.log&#40;10&#41;);
      &#125;
 
      function onChange&#40;) &#123;
        var wins = $('#wins').val&#40;);
        var losses = $('#losses').val&#40;);
        var draws = $('#draws').val&#40;);
 
        wins = 1.0 * &#40;wins || 0&#41;;
        losses = 1.0 * &#40;losses || 0&#41;;
        draws = 1.0 * &#40;draws || 0&#41;;
        var total = wins + losses + draws;
        var winRatio = &#40;wins + &#40;draws / 2.0&#41;) / total;
        var lossRatio = 1 - winRatio;
        var denom99 = 2.58 * Math.sqrt&#40;&#40;winRatio * lossRatio&#41; / &#40;total - 1&#41;);
        var denom95 = 1.96 * Math.sqrt&#40;&#40;winRatio * lossRatio&#41; / &#40;total - 1&#41;);
        var eloWin = elo&#40;winRatio&#41;;
 
        var results = '';
        results += 'ELO&#58; ' + eloWin.toFixed&#40;2&#41; + ' ';
        results += ' +- 99%&#58; ' + &#40;elo&#40;winRatio + denom99&#41; - eloWin&#41;.toFixed&#40;2&#41;;
        results += ' 95%&#58; ' + &#40;elo&#40;winRatio + denom95&#41; - eloWin&#41;.toFixed&#40;2&#41; + '\n';
        results += 'LOS&#58; ' + los&#40;wins, losses&#41;.toFixed&#40;2&#41; + '%\n';
        results += 'Wins&#58; ' + wins + ' Losses&#58; ' + losses + ' Draws&#58; ' + draws + '\n';
        $('#results').text&#40;results&#41;;
      &#125;
      $&#40;document&#41;.ready&#40;function&#40;) &#123;
        $('#wins').keyup&#40;onChange&#41;;
        $('#losses').keyup&#40;onChange&#41;;
        $('#draws').keyup&#40;onChange&#41;;
      &#125;);
    </script>
  </head> 
  <body>
    <h1>ELO ratings</h1>
    <div>
      <div>Wins&#58; <input id="wins"></div>
      <div>Losses&#58; <input id="losses"></div>
      <div>Draws&#58; <input id="draws"></div>
      <pre id="results">
      </pre>
    </div>
  </body> 
</html>

Try to compare if you do not have bugs in your code (I do not know if WhoIsBest works well but I suppose it). If you do not manage to make it work well, I recommend you again to give a try to the last equation of this post (I do not know if it is the same as WhoIsBest or not). Good luck!

Regards from Spain.

Ajedrecista.

[gladius]

Indeed - I had used WhoIsBest incorrectly. Thanks for finding the link btw! An off by one was causing the error. It is fixed now.

[gladius]

So I get

Code: Select all

elo = +13.3
drawelo = 310

The calculation of the LOS is trivial: just mean and stdev and gaussian inverse distribution distribution. I get:

Code: Select all

mu = E(Xi) = 50.9%
sigma = sqrt(V(Xi)/N) = 1.31%
LOS = Phi^-1[(mu-.5)/sigma] = 76.5%

Interesting! So, this is giving more credit per draw essentially?

[Ajedrecista]

Hello Gary:

Indeed - I had used WhoIsBest incorrectly. Thanks for finding the link btw! An off by one was causing the error. It is fixed now.

I am glad to see that LOS problem is fixed... but I just realized about other bug that is much easier to solve: I see that your tool accepts any number (even negative), so you can add if statements or whatever solution to restrict input numbers to positive and zero. Here are some ugly outputs:

Code: Select all

ELO: -279.59  +- 99%: 576.27 95%: 413.07
LOS: 0.00%
Wins: -1 Losses: 1 Draws: 3

Code: Select all

ELO: -Infinity  +- 99%: NaN 95%: NaN
LOS: 0.00%
Wins: -1 Losses: 1 Draws: 2

Code: Select all

ELO: 190.85  +- 99%: NaN 95%: NaN
LOS: 100.00%
Wins: 0 Losses: -1 Draws: 3

Code: Select all

ELO: NaN  +- 99%: NaN 95%: NaN
LOS: 100.00%
Wins: 0 Losses: -1 Draws: 1

Code: Select all

ELO: NaN  +- 99%: NaN 95%: NaN
LOS: 50.00%
Wins: 1 Losses: 1 Draws: -2

There are infinite samples like those ones.

Sorry for not giving my reports joined in one post... I simply did not realize about negative numbers until now!

Regards from Spain.

Ajedrecista.

[Ajedrecista]

Hello again:

Sorry, but I forgot to include a thing when I wrote my previous post in this thread: if you input large numbers (for example 3000, 8000, etc.) in 'Wins' and 'Loses' (typical numbers in GitHub when Marco tests changes in SF), then the tool seems to freeze and I get several messages of 'do you want to stop this script?' if I do not stop the script. I do not know how many times should I close this window for getting the correct result.

I do not know if it is reproducible by anyone else; if that, then the algorithm may be a little inefficient with big numbers of wins and loses. I experienced something like overflow when I wrote my Fortran 95 tool, so I did trial and error and I realized that when I wrote wins + loses > limit (in my case, limit was around 16500 or 17000, so I choosed 16000 for being safe) the programme did not work. So, I can do LOS by Rémi's method up to wins + loses = 16000 in reasonable time (I have to approximate two integrals by Simpson's composite rule (doing 100000 or 200000 partitions, I do not remember which number is the correct): the worst case (big numbers of wins and loses below the upper bound of 16000) takes something less than 0.4 seconds in my Intel Pentium D930 (3 GHz), using single core or even less (I am not totally sure) in Windows XP 32-bit.

Regards from Spain.

Ajedrecista.