CCRL update (23rd May 2008)

[Graham Banks]

The May 23rd update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

The list usually gets updated once or twice during the week and these updates can be viewed here:
http://www.computerchess.org.uk/ccrl/4040.live/
However, no game downloads are available from this live link.

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our standard testing is at 40 moves in 40 minutes repeating and our blitz testing is at 40 moves in 4 minutes repeating, both adjusted to the AMD64 X2 4600+ (2.4GHz).
We have abandoned our 40/12 testing and the link will be removed when Kirill finds the time to do so.

Currently active testers are:
Graham Banks, Shaun Brewer, Kirill Kryukov, Tom Logan, Charles Smith, George Speight, Chuck Wilson and Gabor Szots.
A few other testers are currently taking a break, but remain on our team.


40/40 Notes

Be aware that in the early stages of testing, an engine's rating can often fluctuate a lot.
It is strongly advised to also look at the many other rating lists available in order to get a more accurate overall picture of an engine's rating relative to others.


4CPU 64-bit Engines

Since our last update, Toga II 1.4.1SE, Naum 3.1, Hiarcs Paderborn 2007, Deep Shredder 11 and Glaurung 2.1 have all borne the brunt of our testing.

Naum 3.1 is neck and neck with Zappa Mexico II as second top engine behind Rybka 2.3.2a.

Toga II 1.4.1SE is very evenly matched with Deep Shredder 11, a further 50 elo back.

A narrow margin of 20 elo takes us back to Hiarcs Paderborn 2007, which has made a good start and is currently ahead of both its more vaunted successor Hiarcs 12 and the more elderly Deep Fritz 10.1.
Glaurung 2.1 is also right in there amongst this little group, but it is still early days and things could easily change.

Loop M1-T and Bright 0.3a come in next, holding a clear advantage over Deep Junior 10 and Deep Sjeng 2.7.

Scorpio 2.0 is the only other well tested most recent engine version in this category and brings up the rear.


2CPU Engines

With the emphasis of our multi-cpu testing on 4CPU as opposed to 2CPU, there are gaps in this list and some of the engines also require further games.
We would welcome applications from further testers prepared to focus on 2CPU, so that we could present a more worthwhile rating list in this category.
However, in all honesty, the relative ratings of the engines in this category is pretty much the same as in the 4CPU list.

Not too much action to report since last time with Zappa Mexico II, Naum 3.1, Hiarcs 12 and Toga II 1.4.1SE still in the early stages of testing.


Single CPU Engines

A number of recent engines are amongst those currently being tested.
We are focusing on testing the 32-bit version of Naum 3.1 in this category (because previous versions have shown little to gain from 64-bit implementation), and it has made a very promising start to sit comfortably in second spot between Rybka 2.3.2a and Deep Shredder 11 1CPU! Many more games are of course required before any firm assumptions can be made.
Although Glaurung 2.1 is a nice improvement over its predecessor, its current rating should of course be ignored.

The evenly matched pair of Zappa Mexico II and Fritz 11 aren't too far behind Deep Shredder 11 1CPU.
Deep Shredder 11 1CPU is 64-bit as opposed to Shredder 11 which can only be run as a 32-bit engine.

We have started testing Toga II 1.4.1SE with the expectation that it will likely be a little stronger than the impressive Toga II 3.1.2SE.
Hiarcs 12 Sharpen PV On now has close to 600 games under its belt, so we'll now be putting more emphasis on the default settings before moving on to testing the Paderborn 2007 engine.
Hiarcs 12 could well be stronger with its default Sharpen PV Off setting, so it will be interesting to see whether this is confirmed with further testing.

Next come Loop 13.6 and Fruit 2.3.1, with a comfortable gap between them and the closely grouped Deep Sjeng 2.7, Spike 1.2 Turin, Bright 0.3a and Junior 10.
Thinker 5.1d Passive looks likely to be somewhere around the same strength as the latter bunch.

There is a good distance back to Ktulu 8.0, Frenzee Feb08, Chess Tiger 2007.1 and SmarThink 1.00.

Chessmaster 11, Movei 00.8.438 (10 10 10), Booot 4.14.0, Scorpio 2.0 and Alaric 707 and comprise the next group of engines ahead of E.T Chess 13.01.08, SlowChess Blitz WV2.1, Delfi 5.2, Ruffian 2.1.0 and WildCat 8.
We are close to determining which one or two Chessmaster 11 settings that we will test first.


Free Single CPU Engines

All eyes are focused on whether or not Glaurung 2.1 and Toga II 1.4.1SE can further widen the small gap that the earlier Toga II 3.1.2SE seemed to establish over Rybka 1.0.
Both engines will receive much testing over the coming weeks.

Fruit 2.3.1 is back off the pace, but still comfortably ahead of Spike 1.2 Turin and Bright 0.3a.
Thinker 5.1d Passive is also still in the early stages of testing, but looks likely to be similar in strength to these latter two.
Of course, many of us would dearly love Ryan to release a new version of Fruit to further add to the drama at the top of this list!

Naum 2.0 and Frenzee Feb08 are 40+ ELO further back.

Movei 00.8.438 (10 10 10), Booot 4.14.0, Scorpio 2.0 and Alaric 707 come in next, ahead of E.T Chess 13.01.08, SlowChess Blitz WV2.1, Zappa 1.1, Delfi 5.2, WildCat 8 and Pro Deo 1.6b.

We test a very extensive range of amateur engines (currently ranging down to the 2000 ELO level) through a range of tournaments, all of which can be followed in our public forum.
Our aim is of course to ensure that all engines lower on our lists get 200+ games.


Blitz Notes

An enormous amount of work goes into the blitz list and it is well worth a visit. Shaun has updated the blitz lists during the past day or so.

The latest ratings can be found at one of the following links:
http://computerchess.org.uk/ccrl/404/
http://computerchess.org.uk/ccrl/404.live/

Of special interest to some will be the best free 1CPU engines list which is being constructed through a systematic testing approach as mentioned here:
http://kirr.homeunix.org/chess/discussi ... f=7&t=3271


FRC Notes

No further news here yet because Ray has been away on holiday.

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
If engine authors have a new and stable version of their engine that will run under this GUI, they should contact Ray if they wish to see it tested.

Hiarcs 12 comes in third amongst the available engines behind Shredder 11 and Naum 3 (remembering of course that the top engine, Rybka 2.3.2z3, has remained private).

For FRC the best list to look at is the pure list.
http://www.computerchess.org.uk/ccrl/404FRC/


Stats/Presentation Notes

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

A list of games played this week per engine can be found in the update thread in the CCRL public forum.

All games are available for download by engine, by month or by ECO code.
ELO ratings are now saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.

[Tord Romstad]

Hello Graham,

Some of the engines on the CCRL list are listed in orange color. I always thought this meant that the engines were open source, but I see now that Booot, Sloppy and Micromax are all in green color, even though they are open source. Does orange mean something else than open source, or have you simply forgotten these three engines?

Tord

[Graham Banks]

Hello Graham,

Some of the engines on the CCRL list are listed in orange color. I always thought this meant that the engines were open source, but I see now that Booot, Sloppy and Micromax are all in green color, even though they are open source. Does orange mean something else than open source, or have you simply forgotten these three engines?

Tord

Hi Tord,

I'm not sure what happened there, but I'll bring it to Ray's attention to correct. If you spot any other errors, please let me know.

Thanks, Graham.

[pedrox]

DanaSah also, on occasion has been on orange but I think lately in green.

Pedro

[Graham Banks]

DanaSah also, on occasion has been on orange but I think lately in green.

Pedro

Thanks Pedro. I'll get this sorted also.

Regards, Graham.

[Graham Banks]

DanaSah also, on occasion has been on orange but I think lately in green.

Pedro

Pedro - could you please tell me whether all versions of DanaSah on our list were open source?

Regards, Graham.

[Graham Banks]

Hello Graham,

Some of the engines on the CCRL list are listed in orange color. I always thought this meant that the engines were open source, but I see now that Booot, Sloppy and Micromax are all in green color, even though they are open source. Does orange mean something else than open source, or have you simply forgotten these three engines?

Tord

Hi Tord,

do you know whether or not all versions of Booot have been open source?

Regards, Graham.

[Tony Thomas]

Not all versions of those engines are open source, but the latest versions are.. Danasah is Open source after 2.85 if I remember correctly and Booot after 4.13??

[Tord Romstad]

Hello Graham,

Some of the engines on the CCRL list are listed in orange color. I always thought this meant that the engines were open source, but I see now that Booot, Sloppy and Micromax are all in green color, even though they are open source. Does orange mean something else than open source, or have you simply forgotten these three engines?

Tord

Hi Tord,

do you know whether or not all versions of Booot have been open source?

No, I don't know. If it is somehow relevant for the colors on your list, I should perhaps point out that Glaurung was not always open source. I think the first open source version was 0.2.5 (it may also have been 0.2.3 or 0.2.4, I'm not quite sure).

Tord

[pedrox]

Tony is correct, DanaSah is open source from the version 2.85

List 40/4 is OK, but not list 40/40

Pedro