CCRL rating lists updated (20th August 2011)

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

User avatar
Graham Banks
Posts: 44540
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

CCRL rating lists updated (20th August 2011)

Post by Graham Banks »

The purpose of our rating lists is provide engine authors and enthusiasts with a general comparison of engine strength. We also provide other data that could be of interest.
For various reasons, there may be some engines that do not appear on our lists, therefore it is useful to look at other rating lists as well as ours.
The inclusion or exclusion of engines in our lists should not be taken as our group making a statement about their legality or status. However, where there is controversy surrounding an engine in our lists, we have indicated this.
We test chess engines for our own enjoyment and receive no payment for doing so.


The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
http://computerchess.org.uk/ccrl/4040.live/ (40/40)
http://www.computerchess.org.uk/ccrl/404.live/ (40/4)
http://www.computerchess.org.uk/ccrl/404FRC/ (FRC 40/4)

Please note that the three lists are often updated separately to each other.
The links given in each update report will give you the currently up to date lists.

The links to the various rating lists can be found just beneath the default Best Versions list (as in this screenshot). Specific 32-bit rating lists are denoted as such to the right of the default list in each category. The default lists contain the 64-bit engines.

Image

Our 40 moves in 40 minutes repeating and 40 moves in 4 minutes repeating are both adjusted to the AMD64 X2 4600+ (2.4GHz).

Be aware that in the early stages of testing, an engine's rating can often fluctuate a lot.
It is strongly advised to look at the many other rating lists available in order to get a more accurate overall picture of an engine's rating relative to others.

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are available for download by engine, by month or by ECO code, but you will need to remove the .live from the links if you wish to access these (for example http://computerchess.org.uk/ccrl/4040 ). The download databases by month or ECO code are only updated monthly, but the total games database in its entirety is always available.
The current ELO ratings are saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.
gbanksnz at gmail.com
Norm Pollock
Posts: 1076
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: CCRL rating lists updated (20th August 2011)

Post by Norm Pollock »

Hi Graham,

Is there a minimum number of moves for games to be part of the CCRL rating database?

For example, these ridiculously short games are there:

Code: Select all


[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.04.17"]
[Round "223.1.264"]
[White "N2 0.4 64-bit"]
[Black "Muse 0.899b"]
[Result "0-1"]
[ECO "A10"]
[Opening "English opening"]
[PlyCount "12"]
[WhiteElo "2695"]
[BlackElo "2521"]

1. c4 a6 2. c5 d6 3. cxd6 Qxd6 4. Nc3 Nd7 5. Nf3 Qe6 6. d4 Ngf6 0-1

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.08.24"]
[Round "241.1.467"]
[White "Hiarcs 13.1"]
[Black "Protector 1.3.6 64-bit"]
[Result "1/2-1/2"]
[ECO "D00"]
[Opening "Levitsky attack (Queen's bishop attack)"]
[PlyCount "14"]
[WhiteElo "2983"]
[BlackElo "2956"]

1. d4 d5 2. Bg5 c6 3. Nc3 Qb6 4. Na4 Qa5+ 5. Nc3 Qb6 6. Na4 Qa5+ 7. Nc3 Qb6
1/2-1/2

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.10.31"]
[Round "251.1.223"]
[White "PikoSzachy 3.4"]
[Black "Diablo 0.5.1"]
[Result "1/2-1/2"]
[ECO "D00"]
[Opening "Levitsky attack (Queen's bishop attack)"]
[PlyCount "14"]
[WhiteElo "2500"]
[BlackElo "2510"]

1. d4 d5 2. Bg5 c6 3. Nc3 Qb6 4. Na4 Qa5+ 5. Nc3 Qb6 6. Na4 Qa5+ 7. Nc3 Qb6
1/2-1/2

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.13"]
[Round "253.1.589"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A00"]
[Opening "Benko's opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. g3 c5 2. Bg2 g6 3. c3 Bg7 4. d4 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.14"]
[Round "253.1.609"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "B06"]
[Opening "Robatsch (modern) defence"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. e4 g6 2. d4 Bg7 3. Be3 Nf6 4. Nc3 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.17"]
[Round "253.1.659"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A22"]
[Opening "English opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. c4 e5 2. Nc3 Nf6 3. a3 Be7 4. d3 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.18"]
[Round "253.1.669"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A00"]
[Opening "Benko's opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. g3 g6 2. Bg2 Bg7 3. Nc3 c5 4. d3 1-0


User avatar
Graham Banks
Posts: 44540
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: CCRL rating lists updated (20th August 2011)

Post by Graham Banks »

Norm Pollock wrote:Hi Graham,

Is there a minimum number of moves for games to be part of the CCRL rating database?

For example, these ridiculously short games are there:

Code: Select all


[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.04.17"]
[Round "223.1.264"]
[White "N2 0.4 64-bit"]
[Black "Muse 0.899b"]
[Result "0-1"]
[ECO "A10"]
[Opening "English opening"]
[PlyCount "12"]
[WhiteElo "2695"]
[BlackElo "2521"]

1. c4 a6 2. c5 d6 3. cxd6 Qxd6 4. Nc3 Nd7 5. Nf3 Qe6 6. d4 Ngf6 0-1

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.08.24"]
[Round "241.1.467"]
[White "Hiarcs 13.1"]
[Black "Protector 1.3.6 64-bit"]
[Result "1/2-1/2"]
[ECO "D00"]
[Opening "Levitsky attack (Queen's bishop attack)"]
[PlyCount "14"]
[WhiteElo "2983"]
[BlackElo "2956"]

1. d4 d5 2. Bg5 c6 3. Nc3 Qb6 4. Na4 Qa5+ 5. Nc3 Qb6 6. Na4 Qa5+ 7. Nc3 Qb6
1/2-1/2

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.10.31"]
[Round "251.1.223"]
[White "PikoSzachy 3.4"]
[Black "Diablo 0.5.1"]
[Result "1/2-1/2"]
[ECO "D00"]
[Opening "Levitsky attack (Queen's bishop attack)"]
[PlyCount "14"]
[WhiteElo "2500"]
[BlackElo "2510"]

1. d4 d5 2. Bg5 c6 3. Nc3 Qb6 4. Na4 Qa5+ 5. Nc3 Qb6 6. Na4 Qa5+ 7. Nc3 Qb6
1/2-1/2

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.13"]
[Round "253.1.589"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A00"]
[Opening "Benko's opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. g3 c5 2. Bg2 g6 3. c3 Bg7 4. d4 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.14"]
[Round "253.1.609"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "B06"]
[Opening "Robatsch (modern) defence"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. e4 g6 2. d4 Bg7 3. Be3 Nf6 4. Nc3 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.17"]
[Round "253.1.659"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A22"]
[Opening "English opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. c4 e5 2. Nc3 Nf6 3. a3 Be7 4. d3 1-0

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2010.11.18"]
[Round "253.1.669"]
[White "Waxman 2010"]
[Black "Amundsen 0.80 32-bit"]
[Result "1-0"]
[ECO "A00"]
[Opening "Benko's opening"]
[PlyCount "7"]
[WhiteElo "2402"]
[BlackElo "2427"]

1. g3 g6 2. Bg2 Bg7 3. Nc3 c5 4. d3 1-0


Hi Norm,

those games shouldn't be there because there is obviously something wrong with them. We are looking into this. I agree with you that "short" games shouldn't be included. My personal choice would be no games shorter than 15 moves because sometimes engines come out of book at 8 moves (or a little less) and then get into a situation where they decide to go for a threefold repetition.

Cheers,
Graham.

PS - thanks for picking up these anomalies in our lists. It is very helpful.
gbanksnz at gmail.com
Norm Pollock
Posts: 1076
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: CCRL rating lists updated (20th August 2011)

Post by Norm Pollock »

Hi Graham,

I think a short game that has either a checkmate, stalemate, or 3-fold rep should be kept. That is not the case with the above games.

-Norm
User avatar
Graham Banks
Posts: 44540
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: CCRL rating lists updated (20th August 2011)

Post by Graham Banks »

Norm Pollock wrote:Hi Graham,

I think a short game that has either a checkmate, stalemate, or 3-fold rep should be kept. That is not the case with the above games.

-Norm
Yep - something is broken with those games.
gbanksnz at gmail.com
Norm Pollock
Posts: 1076
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: CCRL rating lists updated (20th August 2011)

Post by Norm Pollock »

I have to correct myself. The 2nd and 3rd games above have 3-fold repetitions.
Norm Pollock
Posts: 1076
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: CCRL rating lists updated (20th August 2011)

Post by Norm Pollock »

Graham Banks wrote:
PS - thanks for picking up these anomalies in our lists. It is very helpful.
Hi Graham,

I hope you don't mind hearing about another anomaly in CCRL 40/40. :)

There are 192 games played by Onno 1.0 32-bit that are exactly the same as 192 games played by Onno 1.0 64-bit. Same opponents, same moves, same results.

Here are links to each set of 192 games (btw, games are numbered for convenience):

http://www.mediafire.com/file/76g7a2sex ... o1032n.pgn

http://www.mediafire.com/file/k5353bc0v ... o1064n.pgn

-Norm
User avatar
Graham Banks
Posts: 44540
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: CCRL rating lists updated (20th August 2011)

Post by Graham Banks »

Norm Pollock wrote:
Graham Banks wrote:
PS - thanks for picking up these anomalies in our lists. It is very helpful.
Hi Graham,

I hope you don't mind hearing about another anomaly in CCRL 40/40. :)

There are 192 games played by Onno 1.0 32-bit that are exactly the same as 192 games played by Onno 1.0 64-bit. Same opponents, same moves, same results.

Here are links to each set of 192 games (btw, games are numbered for convenience):

http://www.mediafire.com/file/76g7a2sex ... o1032n.pgn

http://www.mediafire.com/file/k5353bc0v ... o1064n.pgn

-Norm
Thanks Norm. Will get that checked too. Keep 'em coming! :P
gbanksnz at gmail.com
User avatar
Graham Banks
Posts: 44540
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: CCRL rating lists updated (20th August 2011)

Post by Graham Banks »

Both issues pointed out by Norm have now been addressed in the new 40/40 update.
All games less than 14 moves long have been removed from our database and the pgn files for Onno 1.0 games are now correct.
Games less than 14 moves long will also be removed from the database for tomorrow's 40/4 update.
gbanksnz at gmail.com
Norm Pollock
Posts: 1076
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: CCRL rating lists updated (20th August 2011)

Post by Norm Pollock »

Graham Banks wrote:Both issues pointed out by Norm have now been addressed in the new 40/40 update.
All games less than 14 moves long have been removed from our database and the pgn files for Onno 1.0 games are now correct.
Games less than 14 moves long will also be removed from the database for tomorrow's 40/4 update.
Hi Graham,

Good to hear that.

Btw, I want to let you know that I wasn't sleuthing your database looking for errors. I just happened to come across the errors while testing updates to my 40H PGN utilities.

-Norm