CCRL 40/40, 40/4 and FRC lists updated (16th May 2015)

[Graham Banks]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.

40/40 testing this week that I'm aware of will include (with live broadcast port for TLCV noted where applicable)

Knockout Tournament II (continuing 16002)
53rd Amateur Division 1 Tournament (continuing 16001)
53rd Amateur Division 2 Tournament (starting Monday 16053)
Nirvanachess 2.1c 64-bit Gauntlet (continuing 16063)
Komodo 9 64-bit Gauntlet (continuing 16065)
Donna 2.0 64-bit Gauntlet (continuing 16066)
Vajolet2 2.0 64-bit Gauntlet (continuing 16073)
Jellyfish 1.1 64-bit Gauntlet (continuing 16075)
Maverick 1.0 64-bit Gauntlet (continuing 16076)
Two Old Gentlemen 4CPU Tournament (continuing 16083)
Lights On The Horizon 4CPU Tournament (continuing 16084)
Andscacs 0.80 64-bit 4CPU Gauntlet (being run by Nathanael)
Assorted 4CPU Matches (being run by Charles)
Assorted 1CPU Tournaments (being run by Charles)

40/4 testing this week that I'm aware of will include:

Nirvanachess 2.1c 64-bit
Maverick 1.0 64-bit

[lucasart]

SF really seems to have a specific talent for FRC:

• SF6 is only 6 elo [+/-23] ahead of K9 in the 40/4 pure list
• SF6 is 48 elo [+/-21] ahead of K9 in the FRC 40/4 pure list

Difference is definitely more than error bars.

And K9 has an edge at LTC, or a handicap at STC, depending how you look at it. Maybe that's the explanation. FRC draw ratio is half as much as standard chess in these conditions (which is huge). So playing FRC is like playing faster time control in a way, because blunders are more frequent (often in the first few moves).

[Vinvin]

I'd say, the K9's code is optimized for regular chess ...

SF really seems to have a specific talent for FRC:

• SF6 is only 6 elo [+/-23] ahead of K9 in the 40/4 pure list
• SF6 is 48 elo [+/-21] ahead of K9 in the FRC 40/4 pure list

[MikeB]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.

40/40 testing this week that I'm aware of will include (with live broadcast port for TLCV noted where applicable)

Knockout Tournament II (continuing 16002)
53rd Amateur Division 1 Tournament (continuing 16001)
53rd Amateur Division 2 Tournament (starting Monday 16053)
Nirvanachess 2.1c 64-bit Gauntlet (continuing 16063)
Komodo 9 64-bit Gauntlet (continuing 16065)
Donna 2.0 64-bit Gauntlet (continuing 16066)
Vajolet2 2.0 64-bit Gauntlet (continuing 16073)
Jellyfish 1.1 64-bit Gauntlet (continuing 16075)
Maverick 1.0 64-bit Gauntlet (continuing 16076)
Two Old Gentlemen 4CPU Tournament (continuing 16083)
Lights On The Horizon 4CPU Tournament (continuing 16084)
Andscacs 0.80 64-bit 4CPU Gauntlet (being run by Nathanael)
Assorted 4CPU Matches (being run by Charles)
Assorted 1CPU Tournaments (being run by Charles)

40/4 testing this week that I'm aware of will include:

Nirvanachess 2.1c 64-bit
Maverick 1.0 64-bit

Good Stuff. For an exercise I was testing with some command line pgn tools,
I extracted all games where both computer engines were ranked over 3000 and using Remi's Elo calculator - got the following output on the 40/40 list:

Code: Select all

ResultSet-EloRating>ratings
Rank Name                                  Elo    +    - games score oppo. draws 
   1 Komodo 9 64-bit 4CPU                  215   24   24   506   68%   107   55% 
   2 Stockfish 6 64-bit 4CPU               201   17   17  1060   69%    86   54% 
   3 Komodo 8 64-bit 4CPU                  192   16   16  1298   70%    65   49% 
   4 Stockfish 5 64-bit 4CPU               176   15   15  1324   68%    65   54% 
   5 Houdini 4 64-bit 4CPU                 159   13   13  1722   61%    87   50% 
   6 Stockfish DD 64-bit 4CPU              142   17   17   992   63%    61   56% 
   7 Houdini 4 Tactical 64-bit 4CPU        140   25   25   470   66%    45   52% 
   8 Komodo 9 64-bit                       140   33   33   280   69%    21   49% 
   9 Houdini 3 64-bit 4CPU                 136   13   13  1889   67%    28   48% 
  10 Komodo 7a 64-bit 4CPU                 136   18   18   897   61%    71   59% 
  11 Stockfish 6 64-bit                    133   21   21   724   71%    -1   48% 
  12 Komodo TCEC 64-bit 4CPU               128   18   18   885   62%    50   55% 
  13 Komodo 6 64-bit 4CPU                  117   16   16  1065   59%    65   57% 
  14 Stockfish 4 64-bit 4CPU               112   16   16  1054   61%    47   59% 
  15 Komodo 8 64-bit                       111   18   18   939   69%    -9   50% 
  16 Houdini 3 32-bit                      105   24   24   540   66%     4   47% 
  17 Komodo 5.1 64-bit 4CPU                105   20   20   715   62%    33   55% 
  18 Fire 4 64-bit 4CPU                    104   19   19   735   48%   114   63% 
  19 Stockfish 5 64-bit                    100   16   16  1174   67%    -3   52% 
  20 Houdini 2.0c 64-bit 4CPU               98   16   16  1247   64%     7   45% 
  21 Gull 2.8b 64-bit 4CPU                  94   21   21   656   52%    79   54% 
  22 Houdini 1.5a 64-bit 4CPU               88   21   21   716   63%     6   45% 
  23 Gull 3 64-bit 4CPU                     87   16   16  1126   51%    78   59% 
  24 Houdini 4 64-bit                       78   14   14  1647   61%     6   48% 
  25 Equinox 3.20 64-bit 4CPU               78   22   22   569   50%    77   67% 
  26 Komodo 7a 64-bit                       77   22   22   630   63%    -3   50% 
  27 Stockfish 3 64-bit 4CPU                71   20   20   661   56%    36   58% 
  28 Stockfish DD 64-bit                    68   16   16  1171   62%    -2   54% 
  29 Equinox 3.30 64-bit 4CPU               67   24   24   462   44%   103   61% 
  30 Critter 1.6a 64-bit 4CPU               65    9    9  3134   53%    48   59% 
  31 Stockfish 1.9.1 32-bit                 64   78   78    42   52%    53   62% 
  32 Houdini 2.0c 32-bit                    64   19   19   783   59%     6   49% 
  33 Houdini 1.5a 32-bit                    60   20   20   709   60%     1   51% 
  34 Houdini 4 Tactical 64-bit              60   25   25   465   63%   -21   52% 
  35 Strelka 5.5 32-bit                     58   21   21   616   58%    11   58% 
  36 Houdini 3 64-bit                       58   14   14  1579   61%   -11   49% 
  37 Fire 3.0 64-bit 4CPU                   57   17   17   961   50%    58   58% 
  38 Stockfish 2.2.2 64-bit 4CPU            54   17   17  1037   58%     3   54% 
  39 Rybka 4 32-bit                         53   51   51   104   55%    25   56% 
  40 Houdini 3 Tactical 64-bit              52   19   19   801   62%   -22   50% 
  41 Rybka 4 64-bit 4CPU                    52   13   13  1792   52%    43   53% 
  42 Critter 1.4 64-bit 4CPU                51   20   20   678   59%    -4   56% 
  43 Komodo TCEC 64-bit                     51   17   17  1026   60%    -9   54% 
  44 Rybka 4.1 64-bit 4CPU                  50    9    9  3391   51%    47   56% 
  45 Critter 1.2 64-bit 4CPU                50   21   21   666   56%    12   55% 
  46 Gull 2.3 64-bit 4CPU                   49   17   17   982   47%    64   59% 
  47 Fire 4 64-bit                          49   18   18   822   57%    10   58% 
  48 Bouquet 1.8 64-bit 4CPU                49   15   15  1295   47%    67   59% 
  49 Stockfish 2.3.1 64-bit 4CPU            47   18   18   846   55%    14   52% 
  50 Elektro 1.2 64-bit 4CPU                46   30   30   316   47%    64   57% 
  51 Critter 1.4 32-bit                     43   20   20   684   55%    11   55% 
  52 Strelka 5.1 32-bit                     42   22   22   574   55%    13   61% 
  53 Critter 1.6a 32-bit                    42   21   21   603   55%    11   64% 
  54 Komodo 6 64-bit                        41   19   19   773   58%   -11   52% 
  55 DeepSaros 4.1.6 64-bit 4CPU            38   30   30   319   45%    69   56% 
  56 Sting SF 4.8.4 64-bit 4CPU             37   29   29   316   53%    20   62% 
  57 Bouquet 1.5 64-bit 4CPU                37   23   23   500   46%    57   64% 
  58 Critter 1.6 32-bit                     37   29   29   318   54%    14   64% 
  59 BlackMamba 2.0 64-bit 4CPU             32   18   18   854   44%    71   57% 
  60 Komodo CCT 64-bit                      30   16   16  1173   57%    -9   54% 
  61 Stockfish 2.0.1 64-bit 4CPU            30   25   25   482   55%    -2   49% 
  62 Vitruvius 1.11C 64-bit 4CPU            24   15   15  1224   53%     7   61% 
  63 Stockfish 4 64-bit                     21   19   19   780   57%   -20   57% 
  64 Gull 3 64-bit                          21   14   14  1445   53%     1   58% 
  65 IvanHoe 9.46h 64-bit 4CPU              19   14   14  1399   51%    11   61% 
  66 Hannibal 1.5 64-bit 4CPU               18   30   30   316   52%     5   58% 
  67 Rybka 3 64-bit 4CPU                    18   21   21   621   59%   -35   52% 
  68 Houdini 1.5a 64-bit                    17   15   15  1323   58%   -31   48% 
  69 RobboLito 0.21Q 64-bit 4CPU            16   22   22   569   51%    11   63% 
  70 Houdini 2.0c 64-bit                    16   18   18   906   55%   -14   47% 
  71 Stockfish 2.1.1 64-bit 4CPU            16   29   29   342   51%    11   50% 
  72 Stockfish 2.3.1 32-bit                 14   22   22   572   50%    13   55% 
  73 Sting SF 3 64-bit 4CPU                 13   21   21   643   48%    26   59% 
  74 Bouquet 1.6 64-bit 4CPU                13   21   21   641   50%    13   61% 
  75 Sting SF 2 64-bit 4CPU                 13   27   27   375   49%    21   58% 
  76 Komodo 5 64-bit                        13   12   12  2085   53%    -5   55% 
  77 Critter 1.2 32-bit                     13   28   28   339   52%     4   57% 
  78 Rybka 3 Human 64-bit 4CPU              12   46   46   134   57%   -34   54% 
  79 Texel 1.05 64-bit 4CPU                 12   28   28   364   47%    32   52% 
  80 Komodo 5 32-bit                        11   21   21   604   49%    17   58% 
  81 Critter 1.6a 64-bit                    10   10   10  2857   53%    -6   60% 
  82 Vitruvius 1.11C 32-bit                  9   20   20   662   49%    14   63% 
  83 Komodo 5.1 64-bit                       8   20   20   668   55%   -21   57% 
  84 Stockfish 1.9.1 64-bit 4CPU             6   30   30   312   49%    14   53% 
  85 Strelka 5.5 64-bit                      6    9    9  3193   51%    -1   59% 
  86 Stockfish 2.0.1 32-bit                  5   50   50   106   50%     8   58% 
  87 Stockfish 1.8 64-bit 4CPU               5   28   28   327   50%     3   62% 
  88 Sting SF 4 64-bit 4CPU                  4   23   23   552   42%    54   49% 
  89 Stockfish 2.2.2 32-bit                  3   20   20   715   48%    18   58% 
  90 Stockfish 1.7.1 64-bit 4CPU             2   23   23   486   50%     2   62% 
  91 Stockfish 3 64-bit                      2   18   18   896   52%   -11   53% 
  92 Komodo 4 64-bit                         0   13   13  1784   51%    -5   53% 
  93 Komodo 4 32-bit                        -3   23   23   513   46%    21   54% 
  94 Naum 4.6 64-bit 4CPU                   -3   18   18   932   43%    41   55% 
  95 Equinox 3.30 64-bit                    -4   19   19   779   48%     9   62% 
  96 Equinox 3.20 64-bit                    -4   22   22   548   51%   -12   59% 
  97 Gull 2.2 64-bit 4CPU                   -5   25   25   440   41%    45   57% 
  98 Rybka 3 64-bit 2CPU                    -6   36   36   225   58%   -52   51% 
  99 Chiron 2 64-bit 4CPU                   -8   16   16  1258   40%    55   46% 
 100 DeepSaros 2.3j 32-bit                  -8   19   19   756   45%    19   62% 
 101 Critter 1.01 64-bit 4CPU               -9   33   33   249   49%    -5   59% 
 102 Critter 1.01 32-bit                    -9   39   39   181   49%    -6   52% 
 103 Rybka 4.1 32-bit                      -10   16   16  1031   46%    12   54% 
 104 Gull 2.8b 64-bit                      -11   21   21   633   49%    -7   58% 
 105 IvanHoe 9.46h 32-bit                  -13   19   19   775   45%    17   62% 
 106 Komodo 3 64-bit                       -14   18   18   891   49%    -9   48% 
 107 RobboLito 0.21Q 32-bit                -14   23   23   482   45%    14   64% 
 108 Bouquet 1.6 32-bit                    -15   23   23   514   45%    15   64% 
 109 Komodo 3 32-bit                       -17   20   20   712   44%    17   50% 
 110 Fire 3.0 64-bit                       -17   14   14  1369   48%    -7   59% 
 111 Stockfish 2.1.1 32-bit                -17   26   26   413   45%    11   50% 
 112 Stockfish 1.7.1 64-bit 2CPU           -17   53   53    95   53%   -27   61% 
 113 Hannibal 1.4b 64-bit 4CPU             -17   16   16  1192   39%    49   53% 
 114 Bouquet 1.5 32-bit                    -18   22   22   544   44%    20   57% 
 115 Rybka 4.1 64-bit                      -19    9    9  3504   49%   -11   56% 
 116 Stockfish 2.2 64-bit                  -19   52   52   101   48%    -6   56% 
 117 Bouquet 1.8 64-bit                    -19   11   11  2067   48%    -8   60% 
 118 Critter 0.90 64-bit 4CPU              -21   32   32   278   45%    12   47% 
 119 Deep Fritz 14 64-bit 4CPU             -21   16   16  1053   41%    34   56% 
 120 Protector 1.7.0 64-bit 4CPU           -22   20   20   764   42%    30   52% 
 121 Critter 1.2 64-bit                    -22   15   15  1236   49%   -14   53% 
 122 Critter 1.4 64-bit                    -24   22   22   564   48%   -16   58% 
 123 Stockfish 2.3.1 64-bit                -26   19   19   748   51%   -34   58% 
 124 Sting SF 2 32-bit                     -27   22   22   572   44%    10   54% 
 125 Protector 1.6.0 64-bit 4CPU           -29   20   20   761   41%    29   49% 
 126 Rybka 4 64-bit                        -31   23   23   531   52%   -43   50% 
 127 Stockfish 2.2.2 64-bit                -32   22   22   556   46%    -9   56% 
 128 Texel 1.04 64-bit 4CPU                -33   21   21   664   42%    17   49% 
 129 Sting SF 4.7 64-bit                   -33   22   22   563   44%     0   54% 
 130 Naum 4.2 64-bit 4CPU                  -36   10   10  2837   40%    27   48% 
 131 DeepSaros 3.0 32-bit                  -37   23   23   538   42%    13   57% 
 132 Sting SF 4.8.4 64-bit                 -37   32   32   280   44%     0   53% 
 133 IvanHoe 9.46h 64-bit                  -37   14   14  1348   48%   -26   64% 
 134 Protector 1.5.0 64-bit 4CPU           -37   19   19   826   39%    30   47% 
 135 Gull 2.3 64-bit                       -38   20   20   704   47%   -19   59% 
 136 RobboLito 0.21Q 64-bit                -38   17   17   889   48%   -24   60% 
 137 BlackMamba 2.0 64-bit                 -38   14   14  1360   44%    -6   56% 
 138 DeepSaros 4.1.6 64-bit                -40   17   17   990   45%    -8   59% 
 139 Bouquet 1.5 64-bit                    -40   19   19   740   45%   -10   57% 
 140 Hiarcs 14 4CPU                        -41   14   14  1501   41%    14   51% 
 141 NirvanaChess 2.0a 64-bit 4CPU         -41   32   32   287   40%    26   53% 
 142 Sting SF 5 64-bit                     -41   29   29   344   43%     0   55% 
 143 DeepSaros 4.1.3c 64-bit               -43   26   26   410   46%   -18   59% 
 144 Bouquet 1.6 64-bit                    -43   19   19   771   46%   -20   62% 
 145 Chiron 1.5 64-bit 4CPU                -49   19   19   806   40%    11   52% 
 146 Chiron 1.1a 64-bit 4CPU               -50   23   23   590   38%    23   49% 
 147 Hannibal 1.5 64-bit                   -51   21   21   668   43%    -8   56% 
 148 Naum 4.1 64-bit 4CPU                  -52   43   43   155   48%   -41   51% 
 149 Stockfish 2.1.1 64-bit                -52   18   18   895   46%   -28   54% 
 150 Komodo 2.03 64-bit                    -52   35   35   239   46%   -30   50% 
 151 Vitruvius 1.11C 64-bit                -52   16   16  1060   44%   -16   57% 
 152 Elektro 1.2 64-bit                    -53   26   26   421   45%   -15   51% 
 153 Sting SF 4.6                          -53   31   31   312   43%   -10   48% 
 154 DeepSaros 3.3b 64-bit                 -53   19   19   735   45%   -22   58% 
 155 Gull R375 64-bit 4CPU                 -55   23   23   516   40%     0   58% 
 156 Sting SF 3 64-bit                     -58   17   17   921   44%   -22   54% 
 157 DeepSaros 3.0 64-bit                  -61   23   23   508   43%   -20   56% 
 158 Deep Junior Yokohama 64-bit 4CPU      -61   21   21   677   36%    26   48% 
 159 Naum 4 64-bit 2CPU                    -61   47   47   133   46%   -37   45% 
 160 Hannibal 1.3 64-bit 4CPU              -61   23   23   545   39%     7   49% 
 161 Komodo 2.03 32-bit                    -63   41   41   182   39%     7   45% 
 162 Deep Fritz 13 4CPU                    -64   17   17  1012   38%    11   50% 
 163 Deep Junior 13 64-bit 4CPU            -65   19   19   862   38%    11   49% 
 164 Stockfish 2.0.1 64-bit                -66   34   34   247   46%   -42   52% 
 165 Rybka 3 64-bit                        -67   19   19   737   46%   -46   54% 
 166 Senpai 1.0 64-bit 4CPU                -68   25   25   447   44%   -32   55% 
 167 Stockfish 1.6.3 64-bit 2CPU           -69   38   38   182   45%   -41   61% 
 168 Sting SF 4.7                          -69   24   24   518   40%    -8   52% 
 169 BlackMamba 1.4 64-bit                 -69   16   16  1162   42%   -21   53% 
 170 Stockfish 1.6.3 64-bit 4CPU           -70   34   34   254   44%   -34   48% 
 171 Critter 0.80 64-bit 4CPU              -71   47   47   130   39%    -9   52% 
 172 Sting SF 4 64-bit                     -71   21   21   621   41%   -19   54% 
 173 Deep Junior 13.3 64-bit 4CPU          -72   20   20   735   39%    -3   46% 
 174 Deep Junior 12.5 64-bit 4CPU          -72   28   28   370   41%   -15   46% 
 175 Gull 2.1 64-bit                       -72   21   21   618   40%   -15   58% 
 176 Naum 4 64-bit 4CPU                    -74   29   29   353   44%   -40   50% 
 177 Chiron 2 64-bit                       -76   16   16  1179   40%   -13   49% 
 178 Critter 1.01 64-bit                   -76   28   28   368   46%   -49   53% 
 179 Spike 1.4 Leiden 4CPU                 -77   13   13  1776   38%    -1   48% 
 180 Texel 1.05 64-bit                     -77   31   31   316   43%   -29   46% 
 181 Deep Shredder 12 64-bit 4CPU          -77   14   14  1579   38%    -1   46% 
 182 Komodo 2.01 64-bit                    -78   28   28   356   43%   -35   50% 
 183 Naum 4.6 64-bit                       -79   19   19   821   38%    -9   53% 
 184 Deep Fritz 14 64-bit 1CPU             -82   17   17  1010   38%    -4   51% 
 185 Sting SF 2 64-bit                     -82   23   23   539   41%   -28   57% 
 186 Deep Shredder 12 64-bit OA Off 4CPU   -85   30   30   328   40%   -26   51% 
 187 Protector 1.7.0 64-bit                -87   20   20   725   39%   -20   53% 
 188 Texel 1.03 64-bit 4CPU                -87   49   49   124   41%   -26   44% 
 189 Stockfish 1.7.1 64-bit                -88   43   43   156   42%   -40   48% 
 190 Critter 0.90 64-bit                   -89   42   42   168   43%   -45   47% 
 191 Gull 2.2 64-bit                       -89   22   22   580   37%   -14   53% 
 192 Stockfish 1.9.1 64-bit                -90   32   32   271   43%   -48   53% 
 193 Hannibal 1.4b 64-bit                  -96   14   14  1588   37%   -16   47% 
 194 Rybka 2.3.2a 64-bit 4CPU              -96   28   28   365   34%     0   52% 
 195 Stockfish 1.8 64-bit                  -99   78   78    48   42%   -49   46% 
 196 Protector 1.6.0 64-bit               -102   21   21   682   36%   -17   51% 
 197 Gull II b2 64-bit                    -111   28   28   393   34%   -17   50% 
 198 BlackMamba 1.2c 64-bit               -111   43   43   158   36%   -29   52% 
 199 Protector 1.4.0 64-bit 4CPU          -112   25   25   531   32%     3   41% 
 200 Rybka 3 Dynamic 64-bit               -141  219  219     4   38%   -88   75%

[ernest]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

In the FRC complete list, the standing

Code: Select all

Stockfish 6 64-bit 3474 +14 −14 74.6% −190.4 31.2% 2100   
 
Stoc 040514 64-bit 3419 +15 −15 79.7% −246.0 23.0% 2100 

is strange, unless the more recent Stockfish has a bug !...

[Modern Times]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

In the FRC complete list, the standing

Code: Select all

Stockfish 6 64-bit 3474 +14 −14 74.6% −190.4 31.2% 2100   
 
Stoc 040514 64-bit 3419 +15 −15 79.7% −246.0 23.0% 2100 

is strange, unless the more recent Stockfish has a bug !...

Why strange ? That Stockfish version is over a year old.

[Isaac]

SF really seems to have a specific talent for FRC:

• SF6 is only 6 elo [+/-23] ahead of K9 in the 40/4 pure list
• SF6 is 48 elo [+/-21] ahead of K9 in the FRC 40/4 pure list

Difference is definitely more than error bars.

I think your conclusion is invalid because you are comparing 4 cores list vs a 1 core list.
If you take the 1 core SF6 and K9 in the 40/4 in standard chess, you get (3314-3256)elo = + 58 elo for SF6 vs K9.
So a +48 elo for SF6 in FRC is less spectacular, hence K9 seems to perform better at FRC than in standard chess.
Of course that's forgetting that SF6 on 1 core doesn't appear in the pure list so I can't make a good comparison either and of course the error bar are huge too, so drawing conclusion isn't really possible.

[ernest]

Stoc 040514
That Stockfish version is over a year old.

Looked at 0405, didn't see 14...