CCRL update (14th April 2007)

[Graham Banks]

The April 14th update of the CCRL Rating Lists and Statistics is now available for viewing at:
http://www.computerchess.org.uk/ccrl/4040/

The links to the various rating lists can be found just beneath the default Best Versions list.

Our standard testing is at 40 moves in 40 minutes repeating while our current blitz testing is at both 40 moves in 4 minutes repeating and 40 moves in 12 minutes repeating, all adjusted to the AMD64 X2 4600+ (2.4GHz).

Our team of testers currently includes:
Graham Banks, Ray Banks, Sarah Bird, Shaun Brewer, Kirill Kryukov, Dom Leste, Tom Logan, Andreas Schwartmann, Joshua Shriver, Charles Smith, George Speight, Chris Taylor, Martin Thoresen, Chuck Wilson and Gabor Szots.

We remain on the lookout for a few more testers to help, so please contact one of us if you're interested.


40/40 Notes

827 games were added to our 40/40 database this week, making a total of 54,569 games.
A big thanks to all testers as usual.

Many engines on our list have few games and in many cases their ratings are likely to fluctuate (markedly for some) until a lot more games are played. Therefore no conclusions should be drawn about their strength yet.
To illustrate this point, when an engine has 200 games played, the error margin is still approximately +/-40 ELO, after 500 games +-25 ELO, after 1000 games +-17 ELO and even after 2000 games there is a +-13 ELO error margin!
This of course highlights the importance of looking at other rating lists that are also available in order to draw comparisons and get a more accurate overall picture.

We are not testing Rybka 2.3 at 40/40, preferring to wait for the bugfixed Rybka 2.3.2 to be released.


Multi-CPU Engines
3105 - Rybka 2.2 64-bit 4CPU
3044 - Zap!Chess Zanzibar 64-bit 4CPU
2978 - Hiarcs 11.1 4CPU
2957 - Naum 2.1 64-bit 4CPU
2931 - Deep Shredder 10 64-bit 4CPU
2929 - LoopMP 12.32 2CPU
2919 - Deep Junior 10 4CPU
2918 - Deep Fritz 10 4CPU
2911 - Loop 13.5 64-bit 2CPU
2858 - Glaurung 1.2.1 64-bit 4CPU
2735 - Pharaon 3.5.1 32-bit 2CPU


Single CPU Engines
3009 - Rybka 2.2 64-bit
2905 - Hiarcs 11.1
2891 - Zap!Chess Zanzibar 64-bit
2879 - Fritz 10
2874 - Shredder 10
2871 - Loop 10.32f
2860 - Toga II 1.2.1a
2858 - Loop 13.5 32-bit
2851 - Spike 1.2 Turin
2840 - Fruit 2.2.1
2840 - Naum 2.1 32-bit (64-bit not tested)
2839 - Junior 10
2805 - Ktulu 8.0
2793 - Chess Tiger 2007
2785 - Scorpio 1.84
2771 - Glaurung 1.2.1 Avalanche 32-bit (64-bit not tested)
2762 - CM9000 Enforcer
2760 - CM10th Xperience
2756 - SmarThink 1.00 32-bit (too few games for 64-bit)
2750 - Scorpio 1.91
2743 - Slow Chess Blitz WV2.1


3rd CCRL Amateur Championship (32-bit 1CPU):
Division 1 - Scorpio 1.91 (champion), Colossus 2006f and Pharaon 3.5.1 (relegated)

Division 2 - Frenzee 3.0, Crafty 21.5, E.T Chess 18.11.05 (promoted), Gosu 0.16, AnMon 5.60, Djinn 0.925x (relegated)

Division 3 - Comet B68, Post Modernist 1016, Amateur 2.82 (promoted), Diablo 0.5.1, Aice 0.99.2, Popochin 2.9, Anechka 0.08, Xpdnt 061120 (relegated)

Division 4 - Homer 2.0, Hamsters 0.2, Arion 1.7 (promoted), Sage 2.2a, Tornado 1.0, Natwarlal 0.14, Averno 0.81, Zeus 1.29 (relegated)

Division 5 - With 24 of the 30 rounds having been played, BBChess 1.10 has opened up a small lead over Adam 3.1 and GreKo 5.2.5. NagaSkaki 4.00, NanoSzachy 2.8, Pupsi 0.18 and EveAnn 1.62 are snapping at the heels of the latter two.
Marvin 1.3.0 and Monarch 1.7 are struggling at the other end of the table.


We have a number of tournaments in progress and most of these can be followed in our public forum.


Blitz Notes

The 40/4 is updated separately to 40/40 with the latest update able to be viewed here:
http://www.computerchess.org.uk/ccrl/40 ... index.html


Multi-CPU Engines
3083 - Rybka 2.3.1 64-bit 2CPU
3021 - Zap!Chess Zanzibar 64-bit 4CPU
2993 - Hiarcs 11.1 4CPU
2946 - Deep Shredder 10 64-bit 4CPU
2943 - Deep Fritz 10 4CPU
2938 - Naum 2.1 64-bit 4CPU
2925 - LoopMP 12.32 2CPU
2918 - Loop 13.5 64-bit 2CPU
2916 - Deep Junior 10 4CPU
2860 - Glaurung 1.2.1 32-bit 4CPU
2752 - Pharaon 3.5.1 2CPU
2735 - Deep Frenzee 3.0 64-bit 2CPU


Single CPU Engines
3024 - Rybka 2.2n 64-bit
2917 - Hiarcs 11.1
2910 - Loop 10.32f (not enough games yet for Loop 13.5)
2875 - Fritz 10
2872 - Toga II 1.3x4
2856 - Shredder 10
2853 - Naum 2.1 64-bit
2849 - Fruit 2.2.1
2843 - Junior 10
2827 - Spike 1.2 Turin
2823 - Ktulu 8.0
2796 - Chess Tiger 2007
2764 - Scorpio 1.91
2761 - CM10th Paralyse
2755 - Glaurung 1.2.1 32-bit (higher than 64-bit!)
2723 - Delfi 5.1
2719 - Alaric 704
2718 - Bright 0.1d
2716 - Pro Deo 1.2
2714 - Slow Chess Blitz WV2.1

Zap!Chess Zanzibar has too few games to be included yet.


FRC Notes

Ray tests only those engines that can play FRC through the Shredder Classic GUI.
The testing of Hamsters 0.2 has now concluded.
For FRC the best list to look at is the pure list, and the ratings there are:

2921 - Hiarcs 11.1
2905 - Shredder 10
2893 - Loop 10.32f
2859 - Spike 1.2 Turin
2858 - Fruit 2.2.1
2811 - Naum 2.1
2781 - Glaurung 1.2.1
2673 - Pharaon 3.5.1
2619 - Ufim 8.02
2607 - Movei 0.08.383
2478 - Hermann 1.9
2390 - Aice 0.99.2
2363 - Hamsters 0.2
2362 - Ayito 0.2.994


Stats/Presentation Notes

The LOS stats to the right hand side of each rating list are "likelihood of superiority" stats. They tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are also available for download through the link given at the top of this post. They can be downloaded by engine or by month.
ELO ratings are now saved in all game databases for those engines that have 150 games or more.

A list of games played this week per engine can be found in the update thread in the CCRL public forum, accessible through the link given at the top of this post.

[Hart]

Just a quick question...

I understand long games like 40/40 are good for books, but is this not a bit of overkill for determining engine strength? I see that an engines performance at 40/4 will correlate at roughly .98 with how it will play at 40/40. Might it not be more efficient to scratch 40/40 games altogether and use that time to play (say) 5x more 40/4 games to achieve a rating that is practically no different?

[Graham Banks]

Just a quick question...

I understand long games like 40/40 are good for books, but is this not a bit of overkill for determining engine strength? I see that an engines performance at 40/4 will correlate at roughly .98 with how it will play at 40/40. Might it not be more efficient to scratch 40/40 games altogether and use that time to play (say) 5x more 40/4 games to achieve a rating that is practically no different?

Part of our testing study is to look at such statistics and correlations.
That's why we run 40/4, 40/12 and 40/40.

Regards, Graham.

[Norm Pollock]

Just a quick question...

I understand long games like 40/40 are good for books, but is this not a bit of overkill for determining engine strength? I see that an engines performance at 40/4 will correlate at roughly .98 with how it will play at 40/40. Might it not be more efficient to scratch 40/40 games altogether and use that time to play (say) 5x more 40/4 games to achieve a rating that is practically no different?

Part of our testing study is to look at such statistics and correlations.
That's why we run 40/4, 40/12 and 40/40.

Regards, Graham.

I would prefer to see the 40/40 games limited to 8 book moves rather than 12. They have the "extra" time, so why not use it? Then it would be even better for books, because right now, the engine doesn't begin working until move 13. Of course a game might go out of book before move 12, but that would be an exception. And I would like to see "broader" books, ones that don't exclude a lot of possible opening moves.

[Spock]

I would prefer to see the 40/40 games limited to 8 book moves rather than 12. They have the "extra" time, so why not use it? Then it would be even better for books, because right now, the engine doesn't begin working until move 13. Of course a game might go out of book before move 12, but that would be an exception. And I would like to see "broader" books, ones that don't exclude a lot of possible opening moves.

12 is the maximum. Many of the books used only go to 8 or 10. For example, the book I've been using the last few months is HS-10moves.ctg

So there is a bit of a mixture - 7, 8, 10 and 12 being common

Personally I don't like to go as far as 12, but some testers do