FGRL rating comparison 1 min - 10 min - 60 min

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
fastgm
Posts: 405
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

FGRL rating comparison 1 min - 10 min - 60 min

Post by fastgm » Wed Oct 17, 2018 9:12 pm

Ordo v1.2.6: 3000

1 min + 0.6 sec - http://www.fastgm.de/60-0.60.html

Code: Select all

     Engine              :    Elo   Diff   Error    Points    (%)      W       D      L     D(%)   CFS
 ------------------------------------------------------------------------------------------------------
   1 Stockfish 9         :   3217      0      11    1754.5   77.98    1360    789    101   35.07   100
   2 Houdini 6           :   3181    -36      11    1665.0   74.00    1213    904    133   40.18   100
   3 Komodo 12.1.1       :   3129    -88      10    1524.5   67.76    1047    955    248   42.44   100
   4 Fire 7.1            :   3033   -184      10    1237.0   54.98     693   1088    469   48.36   100
   5 Ethereal 11.00      :   2969   -248       9    1037.5   46.11     509   1057    684   46.98   100
   6 Deep Shredder 13    :   2943   -274      10     956.0   42.49     448   1016    786   45.16   100
   7 Fizbo 2             :   2905   -312      10     838.0   37.24     443    790   1017   35.11    73
   8 Xiphos 0.4          :   2900   -317      10     824.0   36.62     362    924    964   41.07   100
   9 Andscacs 0.94       :   2866   -351      10     722.5   32.11     310    825   1115   36.67    93
  10 Booot 6.3           :   2855   -362      10     691.0   30.71     258    866   1126   38.49   ---

White advantage = 33.65 +/- 2.60
Draw rate (equal opponents) = 52.14 % +/- 0.59
-------------------------------------------------------------------------------------------------------

10 min + 6 sec - http://www.fastgm.de/10min.html

Code: Select all

     Engine              :    Elo   Diff   Error    Points    (%)      W       D      L     D(%)   CFS
 ------------------------------------------------------------------------------------------------------
   1 Stockfish 9         :   3169      0       9    1983.0   73.44    1329   1308     63   48.44   100
   2 Houdini 6           :   3128    -41       8    1843.0   68.26    1110   1466    124   54.30   100
   3 Komodo 12           :   3107    -62       8    1767.5   65.46    1033   1469    198   54.41   100
   4 Fire 7.1            :   3017   -152       7    1422.5   52.69     584   1677    439   62.11   100
   5 Ethereal 11.00      :   2970   -199       7    1236.5   45.80     421   1631    648   60.41   100
   6 Deep Shredder 13    :   2943   -226       7    1130.5   41.87     349   1563    788   57.89    85
   7 Booot 6.3.1         :   2937   -232       7    1108.5   41.06     314   1589    797   58.85    93
   8 Fizbo 2             :   2929   -240       7    1076.5   39.87     367   1419    914   52.56   100
   9 Andscacs 0.94       :   2911   -258       8    1009.0   37.37     268   1482    950   54.89   100
  10 Xiphos 0.4          :   2888   -281       8     923.0   34.19     239   1368   1093   50.67   ---

White advantage = 38.29 +/- 2.01
Draw rate (equal opponents) = 67.75 % +/- 0.49
------------------------------------------------------------------------------------------------------

60 min + 15 sec - http://www.fastgm.de/60min.html

Code: Select all

     Engine              :    Elo   Diff   Error    Points    (%)      W      D      L     D(%)   CFS
 -----------------------------------------------------------------------------------------------------
   1 Stockfish 9         :   3152      0      12     964.5   71.44    613    703     34   52.07   100
   2 Houdini 6           :   3126    -26      11     918.0   68.00    547    742     61   54.96    88
   3 Komodo 12           :   3117    -35      11     902.0   66.81    521    762     67   56.44   100
   4 Fire 7.1            :   3003   -149      10     683.0   50.59    242    882    226   65.33   100
   5 Ethereal 11.00      :   2975   -177      10     628.0   46.52    185    886    279   65.63   100
   6 Deep Shredder 13    :   2951   -201      10     580.5   43.00    170    821    359   60.81    59
   7 Booot 6.3.1         :   2949   -203      10     577.0   42.74    163    828    359   61.33   100
   8 Andscacs 0.94       :   2922   -230      10     524.5   38.85    139    771    440   57.11    88
   9 Fizbo 2             :   2913   -239      10     506.5   37.52    156    701    493   51.93   100
  10 Xiphos 0.4          :   2891   -261      11     466.0   34.52    101    730    519   54.07   ---

White advantage = 34.53 +/- 2.75
Draw rate (equal opponents) = 70.61 % +/- 0.72
----------------------------------------------

Elo and draw difference: 1 min - 10 min

Code: Select all

     Engine                  Diff   D(%)
 ----------------------------------------
   1 Stockfish 9         :   -48  +13.81
   2 Houdini 6           :   -53  +14.12
   3 Komodo 12           :   -22  +11.97
   4 Fire 7.1            :   -16  +13.75
   5 Ethereal 11.00      :    +1  +13.43
   6 Deep Shredder 13    :     0  +12.73
   7 Booot 6.3.1         :   -82  +20.36
 7/8 Fizbo 2             :   +24  +17.45
   9 Andscacs 0.94       :   +45  +18.22
8/10 Xiphos 0.4          :   -12   +9.60
                                  -------
                                  +14.54
Elo and draw difference: 10 min - 60 min

Code: Select all

     Engine                  Diff   D(%)
 ----------------------------------------
   1 Stockfish 9         :   -17   +3.63
   2 Houdini 6           :    -2   +0.66
   3 Komodo 12           :   +10   +2.03
   4 Fire 7.1            :   -14   +3.22
   5 Ethereal 11.00      :    +5   +5.22
   6 Deep Shredder 13    :    +8   +2.92
   7 Booot 6.3.1         :   +12   +2.48
 9/8 Andscacs 0.94       :   +11   +2.22
 8/9 Fizbo 2             :   -16   -0.63
  10 Xiphos 0.4          :    +3   +3.40
                                  -------
                                   +2.52
Elo difference 1st place - 10th place:

Code: Select all

 1 min: 362
10 min: 281
60 min: 261

JJJ
Posts: 1285
Joined: Sat Apr 19, 2014 11:47 am

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by JJJ » Thu Oct 18, 2018 11:32 pm

The difference between 10 min and 60 min is very small, maybe in the error bar range. You could almost stop your 60 min list.

fastgm
Posts: 405
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by fastgm » Sat Oct 20, 2018 10:36 am

Yes, that would be a conclusion from the experiment of the three rating lists.
More than 10 or 15 minutes are apparently not necessary.

JJJ
Posts: 1285
Joined: Sat Apr 19, 2014 11:47 am

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by JJJ » Sat Oct 20, 2018 4:05 pm

Also, it could mean Komodo need 10 min time control for time handicap in their match against human without loosing more than 10 elo.

User avatar
Laskos
Posts: 9410
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by Laskos » Sat Oct 20, 2018 8:51 pm

fastgm wrote:
Sat Oct 20, 2018 10:36 am
Yes, that would be a conclusion from the experiment of the three rating lists.
More than 10 or 15 minutes are apparently not necessary.
No, please, if you can, do the tests as you did till now. And if you can, add to downloads the PGN of the top 10 60 seconds list. I analyzed one week or so ago the 10 min and 60 min lists and PGNs, the apparent results might be skewed by the contempt and scaling issues. Aggravated by the possibility that many top engines borrow ideas from SF. On Sim, although most are not even close of being suspicious, the average similarity increased in the last 2-3 years.
Your lists are a sort of reference to me.

fastgm
Posts: 405
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by fastgm » Sun Oct 21, 2018 2:06 pm

Hello Kai,

it was just a consideration. I will continue the 60 min + 15 sec. list.
You can download now the games from the top 10 60 sec + 0.6 sec list.

Andreas

User avatar
Laskos
Posts: 9410
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by Laskos » Sun Oct 21, 2018 5:42 pm

fastgm wrote:
Sun Oct 21, 2018 2:06 pm
Hello Kai,

it was just a consideration. I will continue the 60 min + 15 sec. list.
You can download now the games from the top 10 60 sec + 0.6 sec list.

Andreas
Thanks, Andreas.

And I saw that PGN of 60''+0.6'' has those same engines.

Sometimes I am not posting here the conclusions from your results, but I am always following and analyzing them.

Uri Blass
Posts: 8556
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: FGRL rating comparison 1 min - 10 min - 60 min

Post by Uri Blass » Sun Oct 21, 2018 7:42 pm

fastgm wrote:
Sat Oct 20, 2018 10:36 am
Yes, that would be a conclusion from the experiment of the three rating lists.
More than 10 or 15 minutes are apparently not necessary.
No

This is not the conclusion.

the problem is that your ratio is 1:10 from 1+0.6 to 10+6 but not 1:10 from 10+6 to 60+15

I prefer for comparison 1+0.6 10+6 and 100+60 and if 100+60 is too expensive then it is better to see
1+0.6 6+3.6 and 36+21.6

Post Reply