CCRL update (30th June 2007)

[Graham Banks]

The June 30th update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

Currently active testers in our team are:
Graham Banks, Ray Banks, Shaun Brewer, Kirill Kryukov, Dom Leste, Tom Logan, Andreas Schwartmann, Charles Smith, George Speight, Chris Taylor, Chuck Wilson, Gabor Szots and Martin Thoresen.

A big thanks to all testers as usual for their efforts this week.

We have now added an openings report page (link at bottom of index page). This lists the number of games played by ECO codes with draw percentage and White win percentage.
Clicking on a column heading will sort the list by that column.
The aim is to soon have games downloadable by ECO code.


40/40 Notes

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +/-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.


Multi CPU Engines
The emphasis this week has been on testing Rybka 2.3.2 and Loop M1-T.
Rybka 2.3.2 64-bit 4CPU currently holds pride of place at the top of our list after 331 games played.
Loop M1-T 64-bit 4CPU has also moved slightly ahead of Loop 13.6 after 203 games played.
The 64-bit 2CPU versions of both these engines are very close in strength to the 4CPU versions.
So the current pecking order amongst 4CPU engines remains Rybka, Zap!Chess Zanzibar, Hiarcs, Naum, Loop, Shredder, Fritz, Junior, Glaurung.


Single CPU Engines
The emphasis in testing this week has been on Loop 13.6 and Rybka 2.3.2.
A Strelka 1.0b gauntlet is also in progress.
Rybka 2.3.2 seems a good improvement in strength at this stage, but early ratings tend to be a little inflated, so a lot more games are still required.
Loop 13.6 now has enough games to be able to say that it is also an improvement in strength over earlier versions.
After 187 games played, Deep Sjeng 2.5 seems to be of similar strength to Fruit 2.2.1.
The current pecking order amongst single cpu engines is Rybka, Zap!Chess, Hiarcs, Loop, Strelka, Fritz, Shredder.


Amateur News:
Strelka 1.0b continues to impress and after 205 games is currently the second best free engine behind Rybka 1.0 and ahead of Toga II 1.2.1a and Spike 1.2 Turin.
Glaurung 2 epsilon/5 is a nice improvement over previous versions, but needs further games to stabilise its rating.
Alaric 704 shows a lot of promise and Delfi 5.1 goes from strength to strength.
Other recent releases to impress are Boot 4.13.1 and Petir 4.39.
We test a very extensive range of amateur engines through our Amateur Championship divisions (32-bit 1CPU) plus other tournaments, all of which can be followed in our public forum.
Our aim is of course to ensure that all engines lower on our lists get at least 200 games.


Blitz Notes

The 40/4 is updated separately to 40/40 with the latest update able to be viewed here:
http://computerchess.org.uk/ccrl/404.live/


Multi-CPU Engines (both 4CPU and 2CPU)
3101 - Rybka 2.3.2 64-bit 2CPU
3019 - Zap!Chess Zanzibar 64-bit 4CPU
2993 - Glaurung 2 epsilon/4 64-bit 2CPU (only 41 games)
2990 - Hiarcs 11.1 4CPU
2968 - Loop M1-T 64-bit 4CPU
2945 - Deep Shredder 10 64-bit 4CPU
2944 - Deep Fritz 10 4CPU
2941 - Naum 2.1 64-bit 4CPU
2916 - Deep Junior 10 4CPU
2805 - Scorpio 1.9 2CPU (only 93 games)
2767 - Deep Sjeng 2.5 2CPU
2754 - Pharaon 3.5.1 2CPU
2733 - Deep Frenzee 3.0 64-bit 2CPU
2713 - Crafty 20.14 2CPU (Crafty 21.5 CPU untested)


Single CPU Engines
3024 - Rybka 2.2 64-bit (Rybka 2.3.2 testing just started)
2913 - Hiarcs 11.1
2908 - Loop 10.32f (too few games for Loop 13.6)
2890 - Toga II 1.3 experimental
2883 - Fritz 10
2870 - Loop 13.6 32-bit (only 116 games)
2854 - Shredder 10
2850 - Naum 2.1 64-bit
2849 - Fruit 2.2.1
2843 - Zap!Chess Zanzibar 32-bit (64-bit untested)
2841 - Junior 10
2839 - Strelka 1.0b
2834 - Junior 10.1
2827 - Spike 1.2 Turin
2821 - Ktulu 8.0
2819 - Chess Tiger 2007.1
2773 - Deep Sjeng 2.5 1CPU
2761 - Glaurung 2 epsilon/2 32-bit (epsilon/5 untested)
2759 - CM10th Paralyse
2750 - Scorpio 1.9
2749 - Scorpio 1.91
2737 - Bright 0.1d
2733 - Bright 0.2a
2732 - SmarThink 1.00 32-bit (64-bit untested)
2721 - Alaric 704
2721 - Pro Deo 1.2
2718 - CM10th Default
2718 - Delfi 5.1
2710 - Slow Chess Blitz WV2.1
2709 - Frenzee 3.0 64-bit
2704 - Chiron 0.8.7
2702 - Gandalf 6
2699 - Pharaon 3.5.1
2695 - WildCat 7
2688 - Movei 0.08.423
2682 - SOS 5.1
2675- Pseudo 0.7c
2673 - Ruffian 1.0.5 (Ruffian 2.1.0 untested)
2669 - Petir 4.39
2665 - Aristarch 4.50


FRC Notes

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
For FRC the best list to look at is the pure list.


Stats/Presentation Notes

The LOS stats to the right hand side of each rating list are "likelihood of superiority" stats. They tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

A list of games played this week per engine can be found in the update thread in the CCRL public forum, accessible through the link given at the top of this post.

All games are available for download through the link given at the top of this post. They can be downloaded by engine or by month.
ELO ratings are now saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom list selections now have the option of including or excluding betas, private engines, settings and others.

[ozziejoe]

it is a pity that zapchess is no longer being developed. it is surprisingly close to rybka with four processors, and seems to scale better. maybe it would be almost even on 16 processors?