CCRL 40/4 lists updated (27th July 2013)

[Graham Banks]

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040/ (40/40)
http://www.computerchess.org.uk/ccrl/404/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are updated separately to each other. The 40/40 and 40/4 lists are updated once every two weeks and alternately to each other. The FRC list is updated when a new engine or engine version is being/has been tested.

The links to the various rating lists can be found just beneath the default Best Versions list (as in this screenshot). Specific 32-bit rating lists are denoted as such to the right of the default list in each category. The default lists contain the 64-bit engines.



Our 40 moves in 40 minutes repeating and 40 moves in 4 minutes repeating are both adjusted to the AMD64 X2 4600+ (2.4GHz).

Be aware that in the early stages of testing, an engine's rating can often fluctuate a lot.
It is strongly advised to look at the many other rating lists available in order to get a more accurate overall picture of an engine's rating relative to others.

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are available for download by engine or ECO code. The total games database in its entirety is always available.
The current ELO ratings are saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.


Games submitted for this update
Colossus 2008b -> 953
Protej 0.5.7 -> 803
Stockfish 3 64-bit 4CPU -> 799
Cheese 1.5 64-bit -> 708
FireFly 2.6.0 64-bit 2CPU -> 639
Stockfish 3 64-bit -> 477
SamChess -> 348
Cmcchess 2.5 -> 346
Vicki 0.041a -> 345
Bremboce 0.6.2 -> 341
Tyrell 1.02 -> 340
MrChess 2.1 -> 339
NoonianChess 3206P2 -> 338
Chess4j 1.2 -> 319
DarkFusch 0.9 -> 318
Project Invincible 2.04 -> 318
Virutor Chess 1.1 -> 316
Ziggy 0.7 -> 316
Mint 2.3 -> 316
Arabian Knight 1.51 -> 315
Darky 0.5d -> 315
Ozwald 0.43 -> 314
Toledo nanoChess v15012010 -> 312
Kitteneitor 0.18.1 32-bit -> 312
AdaChess 1.0 -> 312
HeavyChess 0.13 beta -> 311
Pulsar 2009-9b -> 311
FreeChessII 2.17 -> 310
Nanook 0.17 -> 309
Macromix 2.01 -> 309
MSCP 1.6h -> 309
TRex 1.9 -> 308
Zoe 0.1 -> 307
Goyaz 0.007 -> 307
Raffaela 0.1.4 -> 307
Skaki 1.23 -> 307
Safrad 2.1.35.210 -> 306
Tikov 0.6.3 -> 306
Lodocase 240208 -> 305
JARS 1.75a -> 305
NG-Play 8.3 -> 304
SharpChess2 2.6.9 -> 304
BeRoChess 1.00 32-bit -> 301
Dabbaba 7.49 -> 300
Milady 3.25 -> 300
Tristram 3.36 -> 299
MiniChessAI 1.19 -> 299
Marginal 0.1 -> 299
Zoidberg 0.1 -> 298
Bismark 1.4 -> 296
Joanna 2002 1.03 -> 290
KillerQueen 2.0b3 -> 283
Cinnamon 1.1 64-bit -> 279
Apep 1.01 -> 265
Alibaba 1.38 -> 265
QuTeChess 1.0.1t r3 -> 247
Supra 15.0 -> 239
Abulafia 0.61 -> 230
DarkTemplar 0.1 -> 230
Dolphin 1.0 -> 200
Atlas 3.50 64-bit -> 199
Revati 0.4c -> 198
Gringo 1.4.9b -> 193
Houdini 3 64-bit 4CPU -> 100
The Baron 2.23 64-bit -> 75
Tscp 1.81 -> 74
Petir 4.999999 -> 66
Tucano 2.00 64-bit -> 52
Green Light Chess 3.01.2.2 -> 52
nanoSzachy 4.0 64-bit -> 52
Trace 1.37a -> 52
Yace 0.99.87 -> 52
AnMon 5.75 -> 51
Zappa Mexico II 64-bit 4CPU -> 50
Toga II 1.4 beta5c 4CPU -> 50
Spark 1.0 64-bit 4CPU -> 50
Critter 1.6a 64-bit 4CPU -> 50
Deep Shredder 12 64-bit 4CPU -> 50
Houdini 2.0c 64-bit 4CPU -> 50
Naum 4.2 64-bit 4CPU -> 50
Deep Sjeng WC2008 64-bit 4CPU -> 50
Rybka 4.1 64-bit 4CPU -> 50
Deep Sjeng 3.0 64-bit 4CPU -> 50
Thinker 5.4c Inert 64-bit 4CPU -> 50
Protector 1.4.0 64-bit 4CPU -> 50
Deep Junior 13.3 64-bit 4CPU -> 50
Spike 1.4 Leiden 4CPU -> 49
SOS 5.1 -> 44
Aristarch 4.50 -> 43
Atak 6.8 -> 42
Shallow 1.0 64-bit -> 42
Hermann 2.8 64-bit -> 42
RobboLito 0.21Q 64-bit -> 40
Zappa Mexico II 64-bit -> 40
Spark 1.0 64-bit -> 40
Pepito 1.59.2 64-bit -> 40
Hannibal 1.3 64-bit -> 40
Komodo 5.1 MP 64-bit -> 40
Bouquet 1.6 64-bit -> 39
Gerbil 0.2 64-bit -> 39
E.T. Chess 13.01.08 -> 34
Naum 2.0 64-bit -> 34
List 5.12 -> 34
Pseudo 0.7c -> 34
ProDeo 1.6 -> 34
Ruffian 1.0.5 -> 34
Movei 00.8.438 (10 10 10) -> 34
Booot 4.13.1 -> 34
Alaric 707 -> 34
Slow Chess Blitz WV2.1 -> 34
Fruit 2.3 -> 34
Alfil 8.1.1 Optimized -> 34
Cyrano 0.6b17 64-bit -> 34
WildCat 8 -> 34
Glaurung 2.0.1 64-bit -> 34
Zappa 1.1 64-bit -> 34
Scorpio 2.0 32-bit -> 34
Pharaon 3.5.1 -> 34
Hamsters 0.6 -> 34
BugChess2 1.5.2 -> 34
Frenzee Feb08 64-bit -> 34
Zeus 1.29 -> 33
Bright 0.2c -> 32
Mediocre 0.5 -> 32
OBender 3.2.x -> 32
Little Goliath Evolution 3.12 -> 32
Ant 2006-F -> 32
ALChess 1.5b -> 32
Spike 1.2 Turin -> 32
Chispa 4.0.3 -> 32
AliChess 4.25 -> 32
Madeleine 0.2 -> 32
Parrot 07.07.22 -> 32
Anechka 0.08 -> 32
Ufim 8.02 -> 32
Rodin 6.0 -> 32
Esc 1.16 -> 32
Adam 3.3 -> 32
Uralochka 1.1b -> 32
CyberPagno 2.1 -> 32
Rebel Century -> 32
IQ23.003 -> 32
Waxman 2011 -> 32
Rhetoric Lite -> 32
Popochin 3.2 -> 32
Natwarlal 0.14 -> 32
Anaconda 2.0.1 -> 32
Beowulf 2.4a 32-bit -> 32
ChessterfieldCL i5a -> 32
Mephisto Gideon Professional -> 31
Delfi 5.4 -> 31
NG-Play 8.4 -> 31
Critter 1.6a 64-bit -> 31
Vitruvius 1.11C 64-bit -> 30
Protector 1.5.0 64-bit -> 30
Capivara LK 0.08a11 64-bit -> 30
Adamant 1.7 -> 30
Kanguruh 1.93 -> 30
Spike 1.4 Leiden -> 30
SMIRF 178eBC -> 30
Houdini 3 64-bit -> 30
Rybka 4.1 64-bit -> 30
Naum 4.2 64-bit -> 30
Bace 0.45 -> 30
BigLion 2.23x -> 29
Elf 1.3.0 -> 29
Mizar 3.0 -> 29
KnockOut 0.7.1 32-bit -> 29
GChess IV 0.2.0 -> 28
Roce 0.0390 -> 28
SEE 0.6.9 -> 27
Gull 2.1 64-bit -> 27
N2 0.4 64-bit -> 20
Francesca MAD 0.19 -> 20
Tao 5.6 -> 10
Pupsi2 0.09 64-bit -> 10
Sjaak 524 64-bit -> 10
Vajolet 2.03 -> 10
Murka 2 64-bit -> 10
Glass 2.0 64-bit -> 10
Gibbon 2.57a 64-bit -> 10
Soldat III 0.172 -> 10
pikoSzachy 4.0 -> 10
Philou 3.7.1 64-bit -> 10
Abrok 5.0 -> 10
Chenard 2013.02.27 64-bit -> 10
Matheus 2.3 -> 10
AdroitChess 0.4 32-bit -> 10
Cheese 1.4 64-bit -> 10
Gaia 3.5 64-bit -> 10
Sjakk 2.0 64-bit -> 10
ChessKISS 1.7 64-bit -> 10
Amy 0.8 64-bit -> 10
Monarch 1.7 -> 10
Nebula 2.0 64-bit -> 10
Zeta Dva 0210 64-bit -> 10
Quark 2.35 -> 10
Jabba 13032012 -> 10
Cerebro 3.03d -> 10
MadChess 1.2 64-bit -> 10
BSC 3.9 64-bit -> 10
Amyan 1.72 -> 10
Ax 0.8 -> 10
Djinn 0.987 64-bit -> 10
Alarm 0.93.1 -> 10
Snitch 1.6.2 32-bit -> 9

[Norm Pollock]

Engines in 40/40 that play White more than 53.0% or less than 47.0%, with games >200. White has a score of 54.6% in the full database.

Code: Select all

Name	                      #Games	#White	#Black  	%
				
Toga II 1.4.1SE6	            813	    516	    297	  63.47%
Rhetoric 0.10.d	              318	    195	    123	  61.32%
EveAnn 1.67	                  243	    146	     97	  60.08%
EXchess 5.04b 64-bit	        248	    140	    108	  56.45%
Octochess r4741 64-bit	      248	    140	    108	  56.45%
Glass 2.0 64-bit	            315	    175	    140	  55.56%
Atlas 3.25 64-bit	            408	    225	    183	  55.15%
GNU Chess 5.07.153b 64-bit	  374	    206	    168	  55.08%
Dabbaba 6.52	                257	    140	    117	  54.47%
TJchess 1.1 64-bit	          611	    328	    283	  53.68%
Twisted Logic 0.065e35	      273	    146	    127	  53.48%


BigLion 2.23w	                396	    186	    210	  46.97%
Alfil 13.1 64-bit	            314	    147	    167	  46.82%
FireFly 2.5.13 64-bit	        221	    103	    118	  46.61%
Glaurung 2.1 64-bit	          673	    310	    363	  46.06%
ProChess 1.02AD	              278	    128	    150	  46.04%

[Norm Pollock]

In the list above, Toga II 1.4.1SE6 has a net positive of 219 games playing White compared to playing Black. I broke down the 219 difference to games against 4 engines:

Code: Select all

          Name                    #Games   #White   #Black

Fritz 11                             60        2       58
Glaurung 2.1 64-bit                  55        0       55
Naum 3.1 64-bit                      54        0       54
Rybka 2.3.2a 64-bit                  54        0       54
Toga II 1.4.1SE6                    223      221        2

This unfairly increases Toga's Elo at the expense of Fritz, Glaurung, Naum and Rybka.

[Graham Banks]

In the list above, Toga II 1.4.1SE6 has a net positive of 219 games playing White compared to playing Black. I broke down the 219 difference to games against 4 engines:

Code: Select all

          Name                    #Games   #White   #Black

Fritz 11                             60        2       58
Glaurung 2.1 64-bit                  55        0       55
Naum 3.1 64-bit                      54        0       54
Rybka 2.3.2a 64-bit                  54        0       54
Toga II 1.4.1SE6                    223      221        2

This unfairly increases Toga's Elo at the expense of Fritz, Glaurung, Naum and Rybka.

Yes - that's not good.

[Graham Banks]

In the list above, Toga II 1.4.1SE6 has a net positive of 219 games playing White compared to playing Black. I broke down the 219 difference to games against 4 engines:

Code: Select all

          Name                    #Games   #White   #Black

Fritz 11                             60        2       58
Glaurung 2.1 64-bit                  55        0       55
Naum 3.1 64-bit                      54        0       54
Rybka 2.3.2a 64-bit                  54        0       54
Toga II 1.4.1SE6                    223      221        2

This unfairly increases Toga's Elo at the expense of Fritz, Glaurung, Naum and Rybka.

Those games will be removed for the next update. Thanks for taking the time to spot this.

[Modern Times]

Those games will be removed for the next update. Thanks for taking the time to spot this.

Well, BayesELO factors in a white advantage does it not, so I don't expect much change at all. It will be interesting to see. But nevertheless it is right to remove them.

[Norm Pollock]

Those games will be removed for the next update. Thanks for taking the time to spot this.

Well, BayesELO factors in a white advantage does it not, so I don't expect much change at all. It will be interesting to see. But nevertheless it is right to remove them.

The typical high level chess database has a White score of close to 55% and Black therefore is 45%, after counting draws as 1/2-1/2. That's like flipping an "unfair" coin 20 times and expecting to get 11 heads and 9 tails. That's a mighty big difference for White moving first. It doesn't seem right. But that's what it is.

I don't know what Bayeselo does to compensate. Maybe they give a White win a score of 0.98 (Black gets 0.02 for losing), and maybe they give a Black win a score of 1.02 (White gets -0.02 for losing), and maybe they give Black 0.51 (White 0.49) for a draw? But if that is what they do, they are just playing with numbers in a lame attempt to compensate.

[Modern Times]

I have recalculated the ratings. Toga II 1.4.1SE6 decreased by 3 Elo and there were 1 Elo movements on some other engines. So as I suspected, a very small difference, but good to get rid of that anomaly. Fritz, Naum and Rybka were not affected actually as they had a big volume of games with other engines.

[Norm Pollock]

I checked the extent to which BayesElo compensates for playing Black. And I was amazed by how much it compensates.

Here is what I did. Using my "nameColor" tool, I tagged each engine with (wh) when it played White, and with (bl) when it played Black. I then used BayesElo with a seed value of 2500.

Here are the results for the various versions of "toga". As can be seen, the Black player has a comparable Elo even though the Black "score" is substantially lower than the "score" for the White player. With this being the case, it is no wonder that there was no change with the removal of those games.

Code: Select all

          Name                     Games   Elo   Score  :   Win  :  Draw  :  Lose

Toga II 1.1a 32-bit (bl)            533   2612   50.0%  :  169+  :  195=  :  169-
Toga II 1.1a 32-bit (wh)            525   2627   60.2%  :  220+  :  192=  :  113-
Toga II 1.2 32-bit (bl)             229   2626   48.9%  :   67+  :   90=  :   72-
Toga II 1.2 32-bit (wh)             226   2643   60.4%  :   99+  :   75=  :   52-
Toga II 1.2.1a 32-bit (bl)         1400   2645   43.4%  :  331+  :  554=  :  515-
Toga II 1.2.1a 32-bit (wh)         1392   2659   54.4%  :  486+  :  542=  :  364-
Toga II 1.3.1 (bl)                 1478   2679   43.7%  :  325+  :  642=  :  511-
Toga II 1.3.1 (wh)                 1480   2690   54.4%  :  500+  :  610=  :  370-
Toga II 1.3.4 (bl)                  274   2646   41.4%  :   64+  :   99=  :  111-
Toga II 1.3.4 (wh)                  276   2667   54.3%  :   83+  :  134=  :   59-
Toga II 1.4 beta5c (bl)             264   2703   48.9%  :   66+  :  126=  :   72-
Toga II 1.4 beta5c (wh)             268   2694   57.8%  :   94+  :  122=  :   52-
Toga II 1.4 beta5c 2CPU (bl)        272   2744   46.3%  :   70+  :  112=  :   90-
Toga II 1.4 beta5c 2CPU (wh)        274   2772   60.0%  :  106+  :  117=  :   51-
Toga II 1.4 beta5c 4CPU (bl)        497   2785   46.9%  :  121+  :  224=  :  152-
Toga II 1.4 beta5c 4CPU (wh)        502   2783   56.7%  :  168+  :  233=  :  101-
Toga II 1.4.1SE (bl)               1198   2725   45.3%  :  290+  :  505=  :  403-
Toga II 1.4.1SE (wh)               1251   2717   54.2%  :  396+  :  564=  :  291-
Toga II 1.4.1SE 2CPU (bl)           139   2742   34.5%  :   23+  :   50=  :   66-
Toga II 1.4.1SE 2CPU (wh)           139   2762   47.1%  :   38+  :   55=  :   46-
Toga II 1.4.1SE 4CPU (bl)          1042   2810   44.5%  :  254+  :  420=  :  368-
Toga II 1.4.1SE 4CPU (wh)          1050   2799   53.4%  :  336+  :  450=  :  264-
Toga II 1.4.1SE6 (bl)               297   2724   46.1%  :   70+  :  134=  :   93-
Toga II 1.4.1SE6 (wh)               516   2720   52.9%  :  149+  :  248=  :  119-
Toga II 1.4.1SE6 2CPU (bl)           82   2765   47.0%  :   23+  :   31=  :   28-
Toga II 1.4.1SE6 2CPU (wh)           81   2766   55.6%  :   27+  :   36=  :   18-
Toga II 3.0 (bl)                    107   2739   42.1%  :   27+  :   36=  :   44-
Toga II 3.0 (wh)                    105   2779   57.6%  :   37+  :   47=  :   21-
Toga II 3.0 4CPU (bl)                59   2839   44.9%  :   13+  :   27=  :   19-
Toga II 3.0 4CPU (wh)                59   2854   56.8%  :   21+  :   25=  :   13-
Toga II 3.1.2SE (bl)                506   2729   50.6%  :  144+  :  224=  :  138-
Toga II 3.1.2SE (wh)                507   2707   57.7%  :  186+  :  213=  :  108-

[Michel]

I don't know what Bayeselo does to compensate.

It has a parameter "Advantage" (default value 32.8). BE assume that a player with white has an effective elo of Elo+Advantage. When playing black the effective Elo is Elo-Advantage ("Elo" is the computed elo).

Ordo can do this too BTW but the default value of Advantage is 0.0, so you have to set it manually.