Ordo v0.5 (Release)

michiguel · Post by **michiguel** » Tue Aug 28, 2012 5:47 am

and also Ordoprep

https://sites.google.com/site/gaviotach ... rdoprepv05

And readme file
https://sites.google.com/site/gaviotach ... rdo-readme

The previous code was straightforward, too much that it was waaayyy slow. It was optimized and now it is two order of magnitude faster. It was all algorithmic optimization. It goes now through the whole CCRL database in seconds.

This took ~10 seconds in my 2.4 Ghz AMD

Code: Select all

./ordo -p ccrl4040.pgn -o ccrl-rating.txt -W -a 2400

Loading data &#40;2000 games x dot&#41;&#58; 

..................................................|   100k
..................................................|   200k
..................................................|   300k
..................................................|   400k
......|

Total games           560181
White wins            147770
Draws                 154256
Black wins            110385
No result                  0
Unique head to head    15.49%
Reference rating      2400.0 &#40;average of the pool&#41;

phase iteration  deviation resolution
  0       1       85.52741  166.12394
  1      44        2.95365   38.25175
  2     326        0.00100    0.01139
  3     549        0.00000    0.00000
done

Adjusted White Advantage = 33.752441

phase iteration  deviation resolution
  0       0        0.98082   38.55079
  1       5        0.15150    3.73018
  2     323        0.00006    0.00103
  3     452        0.00000    0.00000
done

This is very important because now doing simulations (-s) is doable.

Miguel

michiguel · Post by **michiguel** » Tue Aug 28, 2012 8:26 am

UPDATE: A small bug in the PGN report at loading was fixed. Current version is v0.5.1

Realized about it ~20 minutes after release. Never payed attention to that.

Miguel

Rebel · Post by **Rebel** » Tue Aug 28, 2012 8:35 am

The modern word for bug is service pack

Ajedrecista · Post by **Ajedrecista** » Tue Aug 28, 2012 5:11 pm

Hi Miguel:

Very good work! Congratulations. I have just tried it and it looks good and fast (good optimization!

):

Code: Select all

C&#58;\&#91;...&#93;\ordo-windows-v0.5.1>ordo-win32 -a 0 -p Houdini_2.0c_Plytest_.pgn -o elo.txt -W

Loading data &#40;2000 games x dot&#41;&#58;

............................|

Total games            57792
White wins             21082
Draws                  19400
Black wins             17310
No result                  0
Unique head to head     0.10%
Reference rating         0.0 &#40;average of the pool&#41;

phase iteration  deviation resolution
  0       1      100.56791  160.45575
  1      58        2.05702    4.62648
  2     351        0.00006    0.00013
  3     427        0.00000    0.00000
done


Adjusted White Advantage = 34.423828


phase iteration  deviation resolution
  0       0        0.60932    5.85542
  1      22        0.14007    0.30989
  2     346        0.00001    0.00001
  3     284        0.00000    0.00000
done

This is the result:

Code: Select all

   # ENGINE        &#58; RATING    POINTS  PLAYED    (%)
   1 Houdini_ply17 &#58;  986.5    2565.0    3792   67.6%
   2 Houdini_ply16 &#58;  900.9    3399.0    5792   58.7%
   3 Houdini_ply15 &#58;  815.6    4094.0    8000   51.2%
   4 Houdini_ply14 &#58;  718.7    4082.0    8000   51.0%
   5 Houdini_ply13 &#58;  616.0    4105.0    8000   51.3%
   6 Houdini_ply12 &#58;  505.1    4177.0    8000   52.2%
   7 Houdini_ply11 &#58;  374.0    4203.5    8000   52.5%
   8 Houdini_ply10 &#58;  221.6    4245.5    8000   53.1%
   9 Houdini_ply09 &#58;   33.6    4158.5    8000   52.0%
  10 Houdini_ply08 &#58; -198.6    3792.0    8000   47.4%
  11 Houdini_ply07 &#58; -369.5    3948.5    8000   49.4%
  12 Houdini_ply06 &#58; -542.4    4028.5    8000   50.4%
  13 Houdini_ply05 &#58; -720.1    4033.0    8000   50.4%
  14 Houdini_ply04 &#58; -901.0    4109.0    8000   51.4%
  15 Houdini_ply03 &#58;-1093.7    2328.0    6000   38.8%
  16 Houdini_ply02 &#58;-1346.7     523.5    4000   13.1%

Ratings are less compressed than in BayesElo, as many people have noted since Ordo 0.2 release; I do not find anything wrong with either algorithm, they are just different, that is all! Just for comparison, here is BayesElo output of the same PGN, courtesy of Adam (download link):

Code: Select all

version 0057.2, Copyright &#40;C&#41; 1997-2010 Remi Coulom. 
compiled Apr  5 2012 17&#58;26&#58;01. 
This program comes with ABSOLUTELY NO WARRANTY. 
This is free software, and you are welcome to redistribute it 
under the terms and conditions of the GNU General Public License. 
See http&#58;//www.gnu.org/copyleft/gpl.html for details. 
ResultSet>readpgn Houdini_2.0c_Plytest_.pgn 
57792 game&#40;s&#41; loaded, 0 game&#40;s&#41; with unknown result ignored. 
ResultSet>elo 
ResultSet-EloRating>mm 1 1 
Iteration 100&#58; 0.0102116 
Iteration 200&#58; 0.00213777 
Iteration 300&#58; 0.000505583 
Iteration 400&#58; 0.000122885 
Iteration 500&#58; 3.00645e-005 
00&#58;00&#58;00,00 
ResultSet-EloRating>confidence 0.95 
0.95 
ResultSet-EloRating>ratings 
Rank Name              Elo      Diff     +      -      Games  Score    Oppo.   Draws     Win           W-L-D 
   1 Houdini_ply17    895.88     0.00   7.31   7.31     3792  67.64%   775.44  51.21%  42.04%      1594-256-1942 
   2 Houdini_ply16    816.87   -79.01   5.91   5.91     5792  58.68%   756.40  52.38%  32.49%      1882-876-3034 
   3 Houdini_ply15    738.32   -78.55   5.06   5.06     8000  51.17%   729.33  49.58%  26.39%      2111-1923-3966 
   4 Houdini_ply14    649.52   -88.80   5.09   5.09     8000  51.02%   640.73  47.90%  27.07%      2166-2002-3832 
   5 Houdini_ply13    555.06   -94.46   5.17   5.17     8000  51.31%   543.69  43.85%  29.39%      2351-2141-3508 
   6 Houdini_ply12    452.69  -102.36   5.26   5.26     8000  52.21%   434.14  41.92%  31.25%      2500-2146-3354 
   7 Houdini_ply11    334.23  -118.47   5.42   5.42     8000  52.54%   309.04  36.89%  34.10%      2728-2321-2951 
   8 Houdini_ply10    197.76  -136.46   5.71   5.71     8000  53.07%   161.26  31.74%  37.20%      2976-2485-2539 
   9 Houdini_ply09     30.63  -167.13   5.91   5.91     8000  51.98%     8.32  26.54%  38.71%      3097-2780-2123 
  10 Houdini_ply08   -172.51  -203.15   5.97   5.97     8000  47.40%  -145.57  25.68%  34.56%      2765-3181-2054 
  11 Houdini_ply07   -326.21  -153.70   5.92   5.92     8000  49.36%  -318.37  26.76%  35.98%      2878-2981-2141 
  12 Houdini_ply06   -484.47  -158.25   5.84   5.84     8000  50.36%  -489.54  26.31%  37.20%      2976-2919-2105 
  13 Houdini_ply05   -647.13  -162.67   5.94   5.94     8000  50.41%  -653.43  24.80%  38.01%      3041-2975-1984 
  14 Houdini_ply04   -812.32  -165.18   6.25   6.25     8000  51.36%  -839.98  20.32%  41.20%      3296-3078-1626 
  15 Houdini_ply03   -990.73  -178.42   6.99   6.99     6000  38.80%  -899.01  19.53%  29.03%      1742-3086-1172 
  16 Houdini_ply02  -1237.58  -246.85   9.98   9.98     4000  13.09%  -901.53  11.72%   7.22%       289-3242-469 
ResultSet-EloRating>x 
ResultSet>x

You have done a great job and it is much appreciated. I have some questions, just for curiosity:

· What is the meaning of Unique head to head?
· IIRC, deviation column already existed in Ordo 0.4 but resolution column is new. What is it?
· How is Adjusted White Advantage calculated (useful addition, by the way)? It is unknown for me at first glance, although I guess that its calculation is not easy with pencil and paper, only knowing number of wins, draws and loses.

Once again, thanks for Ordo. I realize about an improvement in quality over 0.4 version and it is meaningful because Ordo 0.4 was already good! I use it sometimes and I must say that it is very easy to use for me, and this fact means a lot.

This software is a must have IMHO.

Regards from Spain.

Ajedrecista.

Daniel Shawul · Post by **Daniel Shawul** » Tue Aug 28, 2012 5:31 pm

Ratings are less compressed than in BayesElo, as many people have noted since Ordo 0.2 release; I do not find anything wrong with either algorithm, they are just different, that is all! Just for comparison, here is BayesElo output of the same PGN, courtesy of Adam (download link):

Please stop making such ridiculous claims! First understand why the scaling is applied which they both do anyway. Now Ordo has added white advantage to the curious. They both do maximum likelihood estimates so algorithm is still subset of bayeselo. It still needs to do a lot more to catch up with Bayeselo. That is the fact. Draw model, prior, LOS etc. Don't make it a popularity contest, by saying this is excellent this is very good blah blah.

Ajedrecista · Post by **Ajedrecista** » Tue Aug 28, 2012 6:14 pm

Hello Shawul:

Daniel Shawul wrote:Please stop making such ridiculous claims! First understand why the scaling is applied which they both do anyway. Now Ordo has added white advantage to the curious. They both do maximum likelihood estimates so algorithm is still subset of bayeselo. It still needs to do a lot more to catch up with Bayeselo. That is the fact. Draw model, prior, LOS etc. Don't make it a popularity contest, by saying this is excellent this is very good blah blah.

I was not claiming that Ordo was better than BayesElo but I also think that it is fair to say that Ordo is not a crap. I already have an excellent opinion about BayesElo, as expected. BayesElo is the main rating software if I am not wrong so it is telltale.

I do not have time for understand the scaling, draw model, prior, etc. There was no intention to make invincible claims, popularity contests or start a war about rating software. Anyway, it seems that it is impossible to have a calm, peaceful TalkChess forum because personal attacks arise here and there, in many topics, and this is the main drawback of this forum from my POV. I am really getting tired of this obnoxious situation and I do not know if I will hold much time as an active member. Just my unimportant thought.

Once said this: moderators, please feel free to censor or delete this post if you think it is best. I will not get annoyed with that decision.

Regards from Spain.

Ajedrecista.

Michel · Post by **Michel** » Tue Aug 28, 2012 6:36 pm

They both do maximum likelihood estimates so algorithm is still subset of bayeselo.

Ordo does not really do maximum likelihood but rather uses something like the method of moments

http://en.wikipedia.org/wiki/Method_of_ ... tistics%29

except of course it does not use moments but the score of each player.

Remi Coulon observed however that for the logistic distribution (without draws if I recall correctly) the method Ordo uses is equivalent to the corresponding ML method without prior. The prior can be easily simulated
by adding draws.

Can someone tell me the correct name of the statistical method Ordo uses? I have tried to Google it but nothing came up.

Daniel Shawul · Post by **Daniel Shawul** » Tue Aug 28, 2012 7:25 pm

Michel wrote:
They both do maximum likelihood estimates so algorithm is still subset of bayeselo.
Ordo does not really do maximum likelihood but rather uses something like the method of moments

http://en.wikipedia.org/wiki/Method_of_ ... tistics%29

except of course it does not use moments but the score of each player.

Remi Coulon observed however that for the logistic distribution (without draws if I recall correctly) the method Ordo uses is equivalent to the corresponding ML method without prior. The prior can be easily simulated
by adding draws.

Can someone tell me the correct name of the statistical method Ordo uses? I have tried to Google it but nothing came up.

Well in that case it is inferior to maximum likelihood estimate as that wiki page mentions. The method has been superseded by ML and is only to be used as a seed for starting ML iterations. It can be used when we don't have have probability distributions but just utility function, but we do have have probablity distirbution here i.e logistic for both.
My objection was to the confusion caused by scaling parameter. It is so wrong to say bayeselo compresses ratings but elostat don't or anything like that. Yes bayeselo has an option for scaling ratings, so does ordo now. The scale is required to compare it to other elo rating systems.

michiguel · Post by **michiguel** » Tue Aug 28, 2012 11:16 pm

Ajedrecista wrote:Hi Miguel:

Very good work! Congratulations. I have just tried it and it looks good and fast (good optimization!

):

Code: Select all

C&#58;\&#91;...&#93;\ordo-windows-v0.5.1>ordo-win32 -a 0 -p Houdini_2.0c_Plytest_.pgn -o elo.txt -W

Loading data &#40;2000 games x dot&#41;&#58;

............................|

Total games            57792
White wins             21082
Draws                  19400
Black wins             17310
No result                  0
Unique head to head     0.10%
Reference rating         0.0 &#40;average of the pool&#41;

phase iteration  deviation resolution
  0       1      100.56791  160.45575
  1      58        2.05702    4.62648
  2     351        0.00006    0.00013
  3     427        0.00000    0.00000
done


Adjusted White Advantage = 34.423828


phase iteration  deviation resolution
  0       0        0.60932    5.85542
  1      22        0.14007    0.30989
  2     346        0.00001    0.00001
  3     284        0.00000    0.00000
done

This is the result:

Code: Select all

   # ENGINE        &#58; RATING    POINTS  PLAYED    (%)
   1 Houdini_ply17 &#58;  986.5    2565.0    3792   67.6%
   2 Houdini_ply16 &#58;  900.9    3399.0    5792   58.7%
   3 Houdini_ply15 &#58;  815.6    4094.0    8000   51.2%
   4 Houdini_ply14 &#58;  718.7    4082.0    8000   51.0%
   5 Houdini_ply13 &#58;  616.0    4105.0    8000   51.3%
   6 Houdini_ply12 &#58;  505.1    4177.0    8000   52.2%
   7 Houdini_ply11 &#58;  374.0    4203.5    8000   52.5%
   8 Houdini_ply10 &#58;  221.6    4245.5    8000   53.1%
   9 Houdini_ply09 &#58;   33.6    4158.5    8000   52.0%
  10 Houdini_ply08 &#58; -198.6    3792.0    8000   47.4%
  11 Houdini_ply07 &#58; -369.5    3948.5    8000   49.4%
  12 Houdini_ply06 &#58; -542.4    4028.5    8000   50.4%
  13 Houdini_ply05 &#58; -720.1    4033.0    8000   50.4%
  14 Houdini_ply04 &#58; -901.0    4109.0    8000   51.4%
  15 Houdini_ply03 &#58;-1093.7    2328.0    6000   38.8%
  16 Houdini_ply02 &#58;-1346.7     523.5    4000   13.1%

Ratings are less compressed than in BayesElo, as many people have noted since Ordo 0.2 release; I do not find anything wrong with either algorithm, they are just different, that is all! Just for comparison, here is BayesElo output of the same PGN, courtesy of Adam (download link):

Code: Select all

version 0057.2, Copyright &#40;C&#41; 1997-2010 Remi Coulom. 
compiled Apr  5 2012 17&#58;26&#58;01. 
This program comes with ABSOLUTELY NO WARRANTY. 
This is free software, and you are welcome to redistribute it 
under the terms and conditions of the GNU General Public License. 
See http&#58;//www.gnu.org/copyleft/gpl.html for details. 
ResultSet>readpgn Houdini_2.0c_Plytest_.pgn 
57792 game&#40;s&#41; loaded, 0 game&#40;s&#41; with unknown result ignored. 
ResultSet>elo 
ResultSet-EloRating>mm 1 1 
Iteration 100&#58; 0.0102116 
Iteration 200&#58; 0.00213777 
Iteration 300&#58; 0.000505583 
Iteration 400&#58; 0.000122885 
Iteration 500&#58; 3.00645e-005 
00&#58;00&#58;00,00 
ResultSet-EloRating>confidence 0.95 
0.95 
ResultSet-EloRating>ratings 
Rank Name              Elo      Diff     +      -      Games  Score    Oppo.   Draws     Win           W-L-D 
   1 Houdini_ply17    895.88     0.00   7.31   7.31     3792  67.64%   775.44  51.21%  42.04%      1594-256-1942 
   2 Houdini_ply16    816.87   -79.01   5.91   5.91     5792  58.68%   756.40  52.38%  32.49%      1882-876-3034 
   3 Houdini_ply15    738.32   -78.55   5.06   5.06     8000  51.17%   729.33  49.58%  26.39%      2111-1923-3966 
   4 Houdini_ply14    649.52   -88.80   5.09   5.09     8000  51.02%   640.73  47.90%  27.07%      2166-2002-3832 
   5 Houdini_ply13    555.06   -94.46   5.17   5.17     8000  51.31%   543.69  43.85%  29.39%      2351-2141-3508 
   6 Houdini_ply12    452.69  -102.36   5.26   5.26     8000  52.21%   434.14  41.92%  31.25%      2500-2146-3354 
   7 Houdini_ply11    334.23  -118.47   5.42   5.42     8000  52.54%   309.04  36.89%  34.10%      2728-2321-2951 
   8 Houdini_ply10    197.76  -136.46   5.71   5.71     8000  53.07%   161.26  31.74%  37.20%      2976-2485-2539 
   9 Houdini_ply09     30.63  -167.13   5.91   5.91     8000  51.98%     8.32  26.54%  38.71%      3097-2780-2123 
  10 Houdini_ply08   -172.51  -203.15   5.97   5.97     8000  47.40%  -145.57  25.68%  34.56%      2765-3181-2054 
  11 Houdini_ply07   -326.21  -153.70   5.92   5.92     8000  49.36%  -318.37  26.76%  35.98%      2878-2981-2141 
  12 Houdini_ply06   -484.47  -158.25   5.84   5.84     8000  50.36%  -489.54  26.31%  37.20%      2976-2919-2105 
  13 Houdini_ply05   -647.13  -162.67   5.94   5.94     8000  50.41%  -653.43  24.80%  38.01%      3041-2975-1984 
  14 Houdini_ply04   -812.32  -165.18   6.25   6.25     8000  51.36%  -839.98  20.32%  41.20%      3296-3078-1626 
  15 Houdini_ply03   -990.73  -178.42   6.99   6.99     6000  38.80%  -899.01  19.53%  29.03%      1742-3086-1172 
  16 Houdini_ply02  -1237.58  -246.85   9.98   9.98     4000  13.09%  -901.53  11.72%   7.22%       289-3242-469 
ResultSet-EloRating>x 
ResultSet>x

You have done a great job and it is much appreciated. I have some questions, just for curiosity:

· What is the meaning of Unique head to head?

If you have
A v B 1-0
C v D 1-0
A v B 0-1
A v B 1/2
B v A 1-0
C v D 1/2

It is equivalent to

A v B 1.5-1.5
B v A 1-0
C v D 1.5-0.5

3 different type of encounters out of 6 games. That is 50%
The lower this number, the faster the calculation will be.

· IIRC, deviation column already existed in Ordo 0.4 but resolution column is new. What is it?

During the iterations, that would be the "increment" in rating points that is being tried. So, when you see that the resolution is 0.001 it means in each step the most the rating would change is 0.001 ELO equivalent points.

· How is Adjusted White Advantage calculated (useful addition, by the way)? It is unknown for me at first glance, although I guess that its calculation is not easy with pencil and paper, only knowing number of wins, draws and loses.

If you plot Performance of the white player vs Delta Rating, it will give you a curve that is shifted (it does not cross 0,0). So, the white advantage is calculated in such a way that the curve goes through 0, 0.

Once again, thanks for Ordo. I realize about an improvement in quality over 0.4 version and it is meaningful because Ordo 0.4 was already good! I use it sometimes and I must say that it is very easy to use for me, and this fact means a lot. This software is a must have IMHO.

Regards from Spain.

Ajedrecista.

Thanks,
Miguel

michiguel · Post by **michiguel** » Tue Aug 28, 2012 11:33 pm

Michel wrote:
They both do maximum likelihood estimates so algorithm is still subset of bayeselo.
Ordo does not really do maximum likelihood but rather uses something like the method of moments

http://en.wikipedia.org/wiki/Method_of_ ... tistics%29

except of course it does not use moments but the score of each player.

Remi Coulon observed however that for the logistic distribution (without draws if I recall correctly) the method Ordo uses is equivalent to the corresponding ML method without prior. The prior can be easily simulated
by adding draws.

Can someone tell me the correct name of the statistical method Ordo uses? I have tried to Google it but nothing came up.

I have no idea what name to give it, but I think it is similar to ML. In fact, please correct me since I am not a mathematician, you cannot do ML because there is no set of parameters that will guarantee a prediction (most likely outcome) that will match the results you have (that is the definition of ML if I am not wrong). What you can do is this: given certain ratings of your opponents, you can estimate your rating by ML. That is, the rating will be the one that will give an expected performance that matches the actual results. What Ordo achieves is that all the individual ratings satisfy this requirement. Can you call that ML? I don't know, because once calculated all the ratings, the expected global performance of those won't necessarily match the results. However, no set of parameters could achieve that. The typical case is A beat B, B beat C, and C beat A, all 55-45.

Miguel

Ordo v0.5 (Release)

Ordo v0.5 (Release)

Re: Ordo v0.5 (Release)

Re: Ordo v0.5 (Release)

Thank you very much!

Re: Thank you very much!

...

Re: Thank you very much!

Re: Thank you very much!

Re: Thank you very much!

Re: Thank you very much!