CCRL update (11th August 2007)

[Graham Banks]

The August 11th update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

The links to the various rating lists can be found just beneath the default Best Versions list.
For example there is a 32-bit Single CPU list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

Currently active testers in our team are:
Graham Banks, Ray Banks, Shaun Brewer, Kirill Kryukov, Dom Leste, Tom Logan, Andreas Schwartmann, Charles Smith, George Speight, Chris Taylor, Chuck Wilson, Gabor Szots and Martin Thoresen.

A big thanks to all testers as usual for their efforts this week.


40/40 Notes

There currently 69,129 games in our 40/40 database.

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +/-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.


Multi CPU Engines

Rybka 2.3.2a 64-bit 4CPU is a small improvement over Rybka 2.2 64-bit 4CPU.
Interestingly, the improvement is greater on 2CPU.

Zap!Chess Zanzibar 64-bit 4CPU is clearly number 2 ahead of Hiarcs 11.1 4CPU.
Hiarcs 11.2 is untested at this stage.

Naum 2.2 64-bit 4CPU has made a strong start, but has only a handful of games to date.
From what we've seen so far, it looks likely to be an improvement over its well established predecessor.

The current ratings for Loop M1-T 64-bit 4CPU and 2CPU suggest that there is little gain from the extra two CPU.

Deep Shredder 10 64-bit 4CPU, Deep Fritz 10 4CPU and Deep Junior 10 4CPU, are off the pace.


Single CPU Engines

Rybka 2.3.2a leads the ratings here as well, although by a slightly larger margin.
It also looks like the 64-bit version could make more difference to strength than with previous versions.

Toga II 1.3 Beta 1 (likely to be released as Toga II 1.3.1) has now taken over second spot ahead of Zap!Chess Zanzibar!
Loop M1-T could well be a threat to both as it gets more games under its belt.

We are still in the early stages of testing Fruit 051103 and Fruit 2.3.1, and it will be interesting to see where they stand in comparison to each other.

Hiarcs 11.1, Fritz 10 and Shredder 10 are the next three in the ranking order.
The controversial Strelka 1.8 looks likely to be close in strength to them.

We are also still in the early stages of testing Naum 2.2 and Hiarcs 11.2.
Both only have a smallish number of games so far and their current ratings should be ignored.

Spike 1.2 Turin, Junior 10 and Deep Sjeng 2.5 are the next group of engines and seem to be very even in strength.
Junior 10.1 is weaker than Junior 10 according to our testing.

Ktulu 8.0 and Chess Tiger 2007.1 are further adrift.
Alaric 707 and Glaurung 2 epsilon/5 could yet turn out to be stronger than both.


Amateur News

The final release of Toga II 1.3 will rival Rybka 1.0 as the strongest free engine.
Of course Fruit 051103 and Fruit 2.3.1 could well surpass both!

Strelka 1.8 is a little stronger than Spike 1.2 Turin.

Both Alaric 707 and Glaurung 2 epsilon/5 are expected to be stronger than the next group of engines - Scorpio 1.91, Delfi 5.1 and SlowChess Blitz WV2.1.

WildCat 7 and Pro Deo 1.2 are further back.

As we make our way down the list, it should be noted that the most recent versions of Booot, DanaSah, Delphil, Hermann, Alfil, Natwarlal and Feuerstein seem to have made good gains over previous versions.
Others to keep an eye on as they get more games are the latest versions of BugChess2, Matacz, Popochin, NanoSzachy and GreKo.

We test a very extensive range of amateur engines through our Amateur Championship divisions (32-bit 1CPU) plus other tournaments, all of which can be followed in our public forum.

Our aim is of course to ensure that all engines lower on our lists get at least 200 games.

The previously mentioned problem getting Matacz 1.3 to function on Athlons has now been addressed. Thanks Maciej.
The latest version of Twisted Logic is untested yet due to reported problems with frequent losses on time.
Counter 0.2 loses on time far too frequently. We haven't tested the latest version yet to see if this problem has been addressed.


Blitz Notes

There are currently 158,560 games in our 40/4 database.

The 40/4 update is usually done separately to our 40/40 update. The most recent update can always be viewed here:
http://computerchess.org.uk/ccrl/404.live/


FRC Notes

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
If engine authors have a new and stable version of their engine that will run under this GUI, they should contact Ray if they wish to see it tested.

Ray has recently tested Naum 2.2, Fruit 2.3, Fruit 051103, Hamsters 0.4 and Hermann 2.0 and all are now included in the ratings.
He is hoping to test a newer version of Movei next.
The improvement in strength of Hamsters 0.4 is particularly noteworthy.

For FRC the best list to look at is the pure list.
http://www.computerchess.org.uk/ccrl/404FRC/


Stats/Presentation Notes

The LOS stats to the right hand side of each rating list are "likelihood of superiority" stats. They tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

A list of games played this week per engine can be found in the update thread in the CCRL public forum, accessible through the link given at the top of this post.
Please note that our forum has been moved and is now much quicker to load and more readily accessible. The link given will redirect you automatically.

All games are available for download through the link given at the top of this post. They can be downloaded by engine or by month.
ELO ratings are now saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page (link at bottom of index page) lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.
Games can now be downloaded by ECO code.

[ArmyBridge]

Nice Job Graham:!: but the link to download the games per month has been removed, and download 49 or 50 Megas is a slow task if you want to update the data base, (I have only 15 Kb/sec of transfer rate) thanks to all tester
Regards

[Graham Banks]

Nice Job Graham:!: but the link to download the games per month has been removed, and download 49 or 50 Megas is a slow task if you want to update the data base, (I have only 15 Kb/sec of transfer rate) thanks to all tester
Regards

Ray says that if you wait 24 hours, he'll upload the files to the backup site:
http://ccrl.org.uk/ccrl/4040/index.html

He'd like to know whether you prefer the commented pgns or bare pgns, because he'll make the one that you prefer available each week.

Regards, Graham.

[Norm Pollock]

Hi Graham,

I have another "nit-picking" issue. It concerns the way you guys "export" commented pgns. This in no way diminishes my very high opinion of the wonderful job you guys do.

Here is the pgn standard I will refer to:

Code: Select all

8.2.2.2: Export format move number indications
There are two export format move number indication formats, one for use appearing immediately before a white move element and one for use appearing immediately before a black move element. A white move number indication is formed from the integer giving the fullmove number with a single period character appended. A black move number indication is formed from the integer giving the fullmove number with three period characters appended. 

All white move elements have a preceding move number indication. A black move element has a preceding move number indication only in two cases: first, if there is intervening annotation or commentary between the black move and the previous white move; and second, if there is no previous white move in the special case where a game starts from a position where Black is the active player. 

There are no other cases where move number indications appear in PGN export format. 

The link is:

http://www.geocities.com/CapeCanaveral/ ... tm#SEC08.2

The way I see it, your exported commented output is like this:

Code: Select all

14. e4 {+1.22/12 81s} Rcd8 {+0.45/14 72s} 

while I think it should be, according to above-mentioned standard, like this:

Code: Select all

14. e4 {+1.22/12 81s} 14 ... Rcd8 {+0.45/14 72s} 

The standard seems to be quite clear: "A black move element has a preceding move number indication only in two cases: first, if there is intervening annotation or commentary between the black move and the previous white move;".

Btw, simply using "pgn-extract" by David Barnes will correct the situation.

-Norm

[Graham Banks]

Hi Norm,

I'll draw your concern to Kirill's attention.

Regards, Graham.

[Kirill Kryukov]

Hi Graham,

I have another "nit-picking" issue. It concerns the way you guys "export" commented pgns. This in no way diminishes my very high opinion of the wonderful job you guys do.

Here is the pgn standard I will refer to:

Code: Select all

8.2.2.2: Export format move number indications
There are two export format move number indication formats, one for use appearing immediately before a white move element and one for use appearing immediately before a black move element. A white move number indication is formed from the integer giving the fullmove number with a single period character appended. A black move number indication is formed from the integer giving the fullmove number with three period characters appended. 

All white move elements have a preceding move number indication. A black move element has a preceding move number indication only in two cases: first, if there is intervening annotation or commentary between the black move and the previous white move; and second, if there is no previous white move in the special case where a game starts from a position where Black is the active player. 

There are no other cases where move number indications appear in PGN export format. 

The link is:

http://www.geocities.com/CapeCanaveral/ ... tm#SEC08.2

The way I see it, your exported commented output is like this:

Code: Select all

14. e4 {+1.22/12 81s} Rcd8 {+0.45/14 72s} 

while I think it should be, according to above-mentioned standard, like this:

Code: Select all

14. e4 {+1.22/12 81s} 14 ... Rcd8 {+0.45/14 72s} 

The standard seems to be quite clear: "A black move element has a preceding move number indication only in two cases: first, if there is intervening annotation or commentary between the black move and the previous white move;".

Btw, simply using "pgn-extract" by David Barnes will correct the situation.

-Norm

Thanks, Norm. It seems you are correct. I added this into our to-do list.

Best,
Kirill

[Kaj Soderberg]

Hi Graham,

The old but once unbeatable (sort of Rybka) Shredder 7.04 suddenly appeared in the list with quite a promising start. It would be fun to see what it would do after some more games. Maybe you can squeeze it in in some tournament?

Best regards,

Kaj

[Graham Banks]

Hi Graham,

The old but once unbeatable (sort of Rybka) Shredder 7.04 suddenly appeared in the list with quite a promising start. It would be fun to see what it would do after some more games. Maybe you can squeeze it in in some tournament?

Best regards,

Kaj

I'll have to see if I still have it first!

Regards, Graham.

[Uri Blass]

Hi Graham,

The old but once unbeatable (sort of Rybka) Shredder 7.04 suddenly appeared in the list with quite a promising start. It would be fun to see what it would do after some more games. Maybe you can squeeze it in in some tournament?

Best regards,

Kaj

I can see that only the 4 cpu version of shredder7.04 appears in the list

single cpu is more interesting for me and
from the cegt list

93 Shredder 9.1 2750 7 7 6859 57.4 % 2698 31.6 %
106 Shredder 7.04 2724 19 19 949 52.9 % 2703 29.7 %

Uri

[rdan1987]

I can see that only the 4 cpu version of shredder7.04 appears in the list

Uri

I didn't knew that Shredder 7.04 had a 4 CPU version.....