CCRL update (1st February 2008)

[Graham Banks]

The February 1st update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

The list gets updated periodically during the week and these updates can be viewed here:
http://www.computerchess.org.uk/ccrl/4040.live/
Please be aware that no game downloads are available from this live link.

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

Currently active testers in our team are:
Graham Banks, Ray Banks, Shaun Brewer, Kirill Kryukov, Dom Leste, Tom Logan, Charles Smith, George Speight, Chris Taylor, Chuck Wilson, Gabor Szots and Martin Thoresen.


40/40 Notes

There are currently 101,681 games in our 40/40 database.

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.


4CPU 64-bit Engines

Rybka 2.3.2a is over 50+ ELO stronger than Zappa Mexico.
It will be interesting to see whether the just released Zappa Mexico II can close the gap further.

Deep Shredder 11 lies 40+ points further back in third spot.

Naum 2.2 comes in fourth, not too far behind Deep Shredder 11, but ahead of Deep Fritz 10.1 and Hiarcs 11.1.
Toga II 1.4 beta5c is currently at the same level as Naum 2.2, but needs more games to confirm this position.

The remaining well tested engines in order of rating are Loop M1-T, Glaurung 2.0.1, Deep Junior 10, Bright 0.2c, Deep Sjeng 2.7 and Scorpio 2.0.
We are still in the early stages of testing Bright 0.3a.

We are eagerly looking forward to the upcoming release of Naum 3, which by all accounts should make things very interesting at the top of the table!


2CPU Engines

With the emphasis of our multi-cpu testing on 4CPU as opposed to 2CPU, there are gaps in this category and some of the engines also require further games.
However, the order of strength is almost identical to the 4CPU list with the engines that have been well tested.


Single CPU Engines

Rybka 2.3.2a has a massive 120+ ELO lead over Fritz 11, Shredder 11 and Zappa Mexico.

30 ELO further back, Naum 2.2, Toga II 1.3.1 and Hiarcs 11.1 have a slight edge over Loop 13.6 and Fruit 2.3.1.
We are still in the early stages of testing the new Toga Versions. Toga II 3.1.2SE has only a small handful of games and Toga II 1.4 beta5c should make its first appearance soon.

20 ELO back again, the next group of engines - Spike 1.2 Turin, Deep Sjeng 2.7, Glaurung 2.0.1 and Junior 10 are very close in strength.

40 ELO lower are Ktulu 8.0, Chess Tiger 2007.1, SmarThink 1.00 and Bright 0.2c.

Scorpio 1.84 is currently the highest rated of the many Scorpio versions we've tested and is on a par with Booot 4.14.0 and Chessmaster 11.
Chessmaster 11 is 40+ ELO stronger than CM10th Default and is also a little stronger than the best CM10th settings.
Ray has started putting various CM11 settings through extensive blitz testing (600 games each against a variety of engines). The best 1CPU setting so far is CM11th Glorfindel which is only 15 ELO above the default settings.

Alaric 707 and Movei 00.8.438 (10 10 10) are a little further back.

The next group of engines includes SlowChess Blitz WV2.1, Delfi 5.2, Frenzee Dec07, Ruffian 2.1.0, WildCat 7, Pro Deo 1.6b and Gandalf 6.
As it gets more games, Pro Deo 1.6b is now looking likely to be a little stronger than Pro Deo 1.2.

We have only just started testing E.T Chess 13.01.08, Alfil 8.1.1 and Learning Lemming 0.24.
Although Learning Lemming is still private, the author has signalled his intention of a future public release, so it will be interesting to see how it fares.


Free Single CPU Engines

Rybka 1.0 64-bit is still the top free engine ahead of Toga II 1.3.1.
It will be interesting to see if the recently released and highly touted Toga II 1.4 beta5c can finally dethrone Rybka.

Fruit 2.3.1 comes in third ahead of the evenly matched pair of Spike 1.2 Turin and Glaurung 2.0.1.

Naum 2.0 is 40+ ELO further back, slightly ahead of Bright 0.2c, Scorpio 1.84 and Booot 4.14.0.

Alaric 707 and Movei 00.8.438 (10 10 10) come in next, ahead of SlowChess Blitz WV2.1, Delfi 5.2, Frenzee Dec07, Zappa 1.1, WildCat 7 and Pro Deo 1.6b.

Some other recent releases worthy of mention are:
Colossus 2007d - continues to go from strength to strength. This latest version is on a par with Pharaon 3.5.1.
Hamsters 0.6 - Alessandro seems to have made a 400 ELO improvement as he has progressed from Hamsters 0.0.6 through to the latest version in roughly two years!
BugChess2 1.5.2 - Francois has also made astounding progress and this latest version is 200+ ELO ahead of BugChess2 1.4.1 which is just over one year old.

We are looking forward to seeing the results of E.T Chess 13.01.08 and Alfil 8.1.1, both of which are purportedly nice improvements over the previous versions.

We test a very extensive range of amateur engines through a range of tournaments, all of which can be followed in our public forum.
Our aim is of course to ensure that all engines lower on our lists get 200+ games.


Blitz Notes

There are currently 239,337 games in our 40/4 database.

The 40/4 update is usually done separately to our 40/40 update.
The latest ratings can be found at one of the following links:
http://computerchess.org.uk/ccrl/404/
http://computerchess.org.uk/ccrl/404.live/

An enormous amount of work goes into the blitz list and it is well worth a visit.

Kirill has a special project where he is constructing a best free 1CPU engines list through a systematic testing approach as mentioned here:
http://www.talkchess.com/forum/viewtopic.php?t=19206


FRC Notes

There are currently 25,200 games in the FRC 40/4 database.

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
If engine authors have a new and stable version of their engine that will run under this GUI, they should contact Ray if they wish to see it tested.

There is nothing new to report this week.

Although Rybka 2.3.2 FRC is in top spot, it is a private engine.
Therefore Shredder 11 is the strongest available FRC engine, an impressive 80 ELO ahead of Hiarcs 11.1 and Naum 2.2.

For FRC the best list to look at is the pure list.
http://www.computerchess.org.uk/ccrl/404FRC/


Stats/Presentation Notes

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

A list of games played this week per engine can be found in the update thread in the CCRL public forum, accessible through the link given at the top of this post.

All games are available for download through the link given at the top of this post. They can be downloaded by engine or by month.
ELO ratings are now saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page (link at bottom of index page) lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.
Games can now be downloaded by ECO code.

[Tony Thomas]

Cant wait to see your results of Zappa Mexico II.