CCRL broken down by ECO code

[Albert Silver]

Looking at CCRL results, it seemed to me that their balance of openings by eco codes was a bit peculiar. I pulled down the games for sf12 1 CPU and ran them through ordo. Two thing that struck me were the over abundance of ECO A games (800) where SF12 really over performed, and how SF12 really underperformed in ECO B (one could really use some more games there).

Looks normal as A has a wider coverage of good openings.
B has pirc (e4 d6), alekhine (e4 Nf6) and modern (e4 g6) which probably does not deserve more coverage for engine vs engine matches.

That isn't strictly true, plus I have always seen the ratings lists as a guide to what engine will serve the user best. Certainly that is my only interest in the end, and ultimately it should reflect what users play. If you break down the stats of online, OTB, and correspondence play, they all have a roughly 20-21% rate of play even today in the last 3 years for ECO A.

That said, a 4-move book will inherently cut out most such openings because they are defined deeper than 4 moves. ECO A is for openings that deviate early to exotic or unusual lines. You need only see how it is classified. 1.e4 e5 2.a6 will be an ECO A line for example. Mainline Ruy Lopez will never make the cut since it only starts later in the game.

For example, what is the Catalan? The Open Catalan, Classical Line begins 1.d4 Nf6 2.c4 e6 3.g3 d5 4.Bg2 dxc4 5.Nf3 Be7 etc.... and those are the starting moves, not even the actual lines. There are lots of philosophies on suite design. My personal one is of variable length that tries to cover the full range of openings, whether 2 moves deep or 12. Enough to define a line. Why? Well I play the KID, and I want to know how an engine plays it and can aid in preparation or analyzing games. All the ratings in the world won't help if it is a dummy when it comes to actual analysis of positions I play.

[carldaman]

@ Albert
I agree with such a flexible approach to testing a wide variety of lines, not bound by a fixed and arbitrary cut off point, and not influenced by how the engines like the lines being tested. I would only exclude forced repetitions and outright busts.

[Graham Banks]

I don't intend ditching the 10 opening books that I use, so I guess the lopsidedness will continue, just as it likely does with the ECO breakdown in most other rating lists.

[MikeB]

We do our best.

It would be interesting to see a similar ECO distribution breakdown of popular online databases like the Chessbase, ChessOK, PlaychessInfinity, ICCF, IECG, ICOfy, etc.

Graham,

I know there is a tendency on this forum to go to the wall and pillory people who should be praised and thanked instead. I definitely appreciate CCRL’s work over the years.

Certainly it’s become more difficult to differentiate engines with the huge number of draws. I really appreciate that you make the games public so that people like me can mine for interesting nuggets like this.

+1 agree.

[carldaman]

Let's not forget that the ECO codes were created decades ago, reflecting current theory at that time.
There is a certain lopsidedness built into ECO codes to begin with, and testers should be careful about mimicking the ECO
in their book usage.