Reflections on the WCCC

[whereagles]

TCEC is far more like a "world championship" than WCCC/WCSC, but I agree it's not easy to do better with just a week's play.

Disagree that weak engines should be taken out. Low number of engines is not good publicity. Number of entrants should be capped by administrative constraints (i.e. run the tournament in 1 or 2 weeks), not by engine strength.

As for tiebreaks, why not SB first and only then do the "penalty shootout" blitz?

And yeah the broadcast and website could use some improvements...

[Harvey Williamson]

TCEC is far more like a "world championship" than WCCC/WCSC, but I agree it's not easy to do better with just a week's play.

Disagree that weak engines should be taken out. Low number of engines is not good publicity. Number of entrants should be capped by administrative constraints (i.e. run the tournament in 1 or 2 weeks), not by engine strength.

As for tiebreaks, why not SB first and only then do the "penalty shootout" blitz?

And yeah the broadcast and website could use some improvements...

Before the blitz there were two 45+15, playoff, games which were both drawn. Holding over 2 weeks would double the cost and is just not practical.

[Leto]

For what it's worth here are some reflections on the WCCC for consideration, planning for next time and discussion.

STUFF I LIKED
* The double round approach was excellent and made the event so much better than on some previous years where there's only been something like 5 rounds and a winner is declared. More games = better.

* The standard was high. In the past this event has been mocked as the "Disabled Olympics" when Junior won (although I'd like to see Junior back) but this year there were some seriously strong entrants and, combined with the custom hardware and opening book, this made it the strongest tournament in the world to date IMO. Not a small feat!

* Not having the Baron. This was a good call, The Baron may be a great project, but it has no place in a WCCC since it's too weak at the moment. Maybe in years to come it might be different, but at the moment scoring only 2 draws in the WCSC says it all.

* We can follow it live.

* The unlimited hardware in the WCCC. Essential in my opinion. To give this credibility, this section MUST be totally open to all entrants, from small computers to custom made supercomputers like Hydra, Deep Blue and the modern way: Clusters. It can't be a WORLD championship if certain entrants can't be allowed in if they are strong enough. I think this also encourages innovation (several programmers have hinted they are looking into cluster options - my lips are sealed of course) and this: I've always held a belief that computer chess thrives when there is a supercomputer around since it inspires us to try harder by giving a vision of the level chess can get to. Supercomputers also draw a crowd. There's something 'sexy' about them, and that's good for the sport.

* The old and the new. Great to see some classics like HIARCS and Shredder, as well as the current champ - Komodo the Dragon - and private engines like Jonny. Great mix.

* The different divisions. WCCC, WCSC, Blitz - they all have their place. Great format.


STUFF I THINK COULD BE IMPROVED
* There is something just a bit disorganized about the website. People often resort to the forums because they can't find links to things like hardware specs, or the time control. It needs reworking and laying out more clearly. The hardware specs - although stated - were clearly incomplete and a lot of helpful information could have been added. For example, it was frustrating that the former hardware guy for the Rybka team and myself eventually worked out the spec of the Komodo workstation pretty much by deduction. Stuff like this could be done so much better.

* The live following of the games. It's great that it's happening, but the viewer left a lot to be desired and the live PGN didn't auto update - can anything be done about this? Could it be broadcast through Chessbase or something? (maybe it was, just asking)

* Reporting back from the event was patchy. One day we'd get a flurry of news and gossip, then nothing. A consistent format would be good.

* The playoff end to the WCCC. For me this was a shame. If we really believe in the title "WORLD COMPUTER CHESS CHAMPION", do we want this to be decided...in a 3 minute game? I know people need to go to bed and all that stuff, but just ask - how would Garry feel if his match with Anatoly was decided by a 3 min game? The title would be degraded. Also I suspect Jonny was handicapped by this time control, but that's another matter.

Just some thoughts...overall a highlight to the chess year though!

Just a note. There HAVE been human WC's crowned as the result of a blitz game. I no longer remember when this was, but within the last 30-40 years where a WC was ended with a series of blitz games. Not my idea of the right way to do it, but I presume it was to deal with the "draw odds" issue that would let the current WC retain the title if the match was drawn after N games.

The Kramnik vs Topalov match is the one you're thinking about. Kramnik won on a sudden death 5 minute blitz game if I remember correctly.

[bob]

TCEC is far more like a "world championship" than WCCC/WCSC, but I agree it's not easy to do better with just a week's play.

Disagree that weak engines should be taken out. Low number of engines is not good publicity. Number of entrants should be capped by administrative constraints (i.e. run the tournament in 1 or 2 weeks), not by engine strength.

As for tiebreaks, why not SB first and only then do the "penalty shootout" blitz?

And yeah the broadcast and website could use some improvements...

The WCCC has always been about compromises. For example:

SSDF, CCRL and such use uniform platform which has advantages and disadvantages. No deep thought / deep blue / belle / hi tech / cray blitz /*socrates /etc. Nobody using the Dec Alpha (when it was the only 64 bit microprocessor around). What about multiple cores? Most are testing with one. What about larger numbers on NUMA platforms? What about GPU assists? So a uniform platform CCRL is pretty exclusionary and ignores a LOT of game tree search research activity. It is non-trivial to use all these oddball architectural varieties efficiently. What if someone has spent years working on an efficient GPU assistance device for their engine? Should that be excluded?

On the other hand, CCRL, SSDF and such weed out the hardware advantages one might have with deep pockets. So you get a sort of "engine-only" measurement but with a number of engines excluded or penalized (i.e. In 1997 Crafty could did run on Intel 32 bit hardware, but it ran significantly faster on the Alpha since it was always a 64 bit application.)

Which is better? Potato? Potatah? Tomato? Tomatah?

(2) format. WCCC has always been a "participants gather and play at one location" to facilitate information exchange and build relationships. CCRL/SSDF doesn't do any of that. At the WCCC there has always been a panel discussion, or a set of presentations, etc, and since the Olympiad came along it grew well beyond computer chess exclusively.

However, the format of WCCC has its own issues. When the WCCC was started, it was a 5 round event. When I won the 1983 WCCC event, we have over 30 entrants in NYC and it was still 5 rounds. Ditto when I won in Cologne in 1986, 5 rounds again. It made the event duration palatable, but if you start to spread the games one per day (We used to play 2 on Sunday, then one on Monday Tuesday and Wednesday, or some variation of that in the early years) then suddenly the event takes a week. With travel, you begin to infringe on two weeks, allowing, at best, one day for travel, then a day to get set up and deal with any hardware issues, communication issues and such at the remote venue. Taking two weeks off from school was really not possible for me. These things used to be held in the Fall and a major computer conference such as ACM or supercomputing. Which fit in well with academia where many of us would present papers at these events. Missing a week was painful but we could get classes covered. Moving to a longer event made this very difficult. And moving it back in time to the Summer rather than Fall did not help for those of us that often ended up teaching full-time in the summer in a CS program.

3. operation. Obviously, today, we can do online events that are quite effective. We have had some hugely successful online events, one CCT drawing over 50 programs. But it is not quite the same as meeting in one place. I could show you a scrapbook I have from 1984 after we won the 1983 WCCC event where there were stories about Cray Blitz in EVERY computer publication known to man, plus things like the New York Times, New Yorker magazine, Science Magazine, Computer World, Byte Magazine, Flight magazine, shoot, the CB team was on the cover of the February 1984 Chess Life and Review (USCF) magazine. We had every major network TV franchise present. Every night. Etc.

CCT had no publicity whatsoever, nor does CCRL and such. WCCC did MUCH to foster interest in computer chess over the years, leading to the IBM Deep Blue project and the ridiculous publicity it generated.

But the online events are about as much fun to me nowadays, and they require no travel or cost, which is a big plus. Since I have retired, I could certainly attend anything I want, but I don't really want to spend two weeks in a place that my family is not so interested in visiting.

4. tie-breaks. WCCC has done it both ways. Originally it was sum of opponents period. This gave Cray Blitz the title in 1986 in Germany. We instituted a playoff game somewhere but now you have the "color" advantage and the one drawing white has an advantage. Then two games to equalize colors but now you can end in a draw. No perfect solution here. But I tend to prefer simplicity. SB. But in the current event you could NOT use SB. In a round robin? Everybody has the same SB score.

So there is no ideal solution here. I'd prefer what we did in the early ACM days and simply have two co-champions at the events back then. If you have 32 players and play 5 rounds, SB is messy as what happens if one of the two eventual winners is paired way too low and gets a weak opponent in the first two rounds as he makes his way up the ranking? His SB is ruined.

The approach used at the WCCC is certainly not the worst I have seen. They played two long playoff games first, then resorted to blitz, exactly as the human WCC was handled years ago.

[whereagles]

Robert, how about a 3-1-0 score for WDL?

For the rest, I agree of course.

[APassionForCriminalJustic]

For what it's worth here are some reflections on the WCCC for consideration, planning for next time and discussion.

STUFF I LIKED
* The double round approach was excellent and made the event so much better than on some previous years where there's only been something like 5 rounds and a winner is declared. More games = better.

* The standard was high. In the past this event has been mocked as the "Disabled Olympics" when Junior won (although I'd like to see Junior back) but this year there were some seriously strong entrants and, combined with the custom hardware and opening book, this made it the strongest tournament in the world to date IMO. Not a small feat!

* Not having the Baron. This was a good call, The Baron may be a great project, but it has no place in a WCCC since it's too weak at the moment. Maybe in years to come it might be different, but at the moment scoring only 2 draws in the WCSC says it all.

* We can follow it live.

* The unlimited hardware in the WCCC. Essential in my opinion. To give this credibility, this section MUST be totally open to all entrants, from small computers to custom made supercomputers like Hydra, Deep Blue and the modern way: Clusters. It can't be a WORLD championship if certain entrants can't be allowed in if they are strong enough. I think this also encourages innovation (several programmers have hinted they are looking into cluster options - my lips are sealed of course) and this: I've always held a belief that computer chess thrives when there is a supercomputer around since it inspires us to try harder by giving a vision of the level chess can get to. Supercomputers also draw a crowd. There's something 'sexy' about them, and that's good for the sport.

* The old and the new. Great to see some classics like HIARCS and Shredder, as well as the current champ - Komodo the Dragon - and private engines like Jonny. Great mix.

* The different divisions. WCCC, WCSC, Blitz - they all have their place. Great format.


STUFF I THINK COULD BE IMPROVED
* There is something just a bit disorganized about the website. People often resort to the forums because they can't find links to things like hardware specs, or the time control. It needs reworking and laying out more clearly. The hardware specs - although stated - were clearly incomplete and a lot of helpful information could have been added. For example, it was frustrating that the former hardware guy for the Rybka team and myself eventually worked out the spec of the Komodo workstation pretty much by deduction. Stuff like this could be done so much better.

* The live following of the games. It's great that it's happening, but the viewer left a lot to be desired and the live PGN didn't auto update - can anything be done about this? Could it be broadcast through Chessbase or something? (maybe it was, just asking)

* Reporting back from the event was patchy. One day we'd get a flurry of news and gossip, then nothing. A consistent format would be good.

* The playoff end to the WCCC. For me this was a shame. If we really believe in the title "WORLD COMPUTER CHESS CHAMPION", do we want this to be decided...in a 3 minute game? I know people need to go to bed and all that stuff, but just ask - how would Garry feel if his match with Anatoly was decided by a 3 min game? The title would be degraded. Also I suspect Jonny was handicapped by this time control, but that's another matter.

Just some thoughts...overall a highlight to the chess year though!

Man you are completely delusional. You are darn right that this tournament is one heck of a disabled Olympic Tournament. Have you looked at the number of participants (six?! Really) - and quality of the participants?! The only reason why I was following the WCCC, and many others, was due to Komodo being in the tournament. The strongest tournament to date? What on earth have you been smoking? You cannot have the strongest tournament(s) WITHOUT the strongest engines or competitors. Hardware in and of itself is not everything. Look at how Jonny get spanked versus Komodo. You made a prediction that Jonny should catch Komodo in ELO alone with merely just having 2400 (2376) cores to use. I thought that was ridiculous since a MUCH stronger engine (world number one) with super strong hardware like what Mark and Larry had (4890 v2s if I am not mistaken) in this tournament should be MORE than enough to surpass the far more INFERIOR engine with thousands of cores. Software is everything; how an engine evaluates a position, and which moves it decides to prune cannot just be dependent on how fast a computers' raw CPU speed is. At 48 cores and great scaling, coupled with this being Komodo, it would have been extremely difficult to beat. Therefore, the fact that it went undefeated really should not be of any surprise. Yes I am sure that you could argue that Komodo's "legend" of an operator with his opening repertoire helped team Komodo a lot too.

Another major issue with the tournament is the fact that there are not nearly enough games to be of any statistical relevance. A true championship should last months on end, so that we can see truly which engines are the strongest. Anyone with a half a brain could write a thesis at a university to prove undoubtedly that this tournament has been rightfully superseded by TCEC.‎ Seriously.
‎
‎In my opinion what this tournament did show is that Komodo truly and unequivocally is the world's number one engine - something that a lot of us enthusiasts have been aware of for a long time. The only engine that might potentially be stronger is Stockfish. That engine is certainly not Jonny, HIARCS, Shredder, or Deep Junior (no disrespect). Komodo went through the entire tournament without a single loss. Thus that result coincides with the many championships that Komodo has won in the TCEC tournament.‎
‎
I think that tournaments are great for the community, and for showcasing a lot of the talent that exists in our world. But a tournament pretending to be something that it is not is downright pathetic. I think that it would be fun to participate in something like the WCCC‎ because it allows for competition over the board. I think that it would be cool to play OTB by relaying the moves distinctively chosen by the engines to the board. Likewise, being able to use custom home-brewed opening books finely tuned would allow for style and choices within the tournament not available in a tournament such as TCEC. Nevertheless, calling the WCCC tournament the strongest tournament ever is a joke man.

[whereagles]

chill out, man.. it's not that important

[Dan Cooper]

I thought the tournament was good given the constraints they're under and different goals than something like TCEC.

The tie-breaker they ended up using was much better than the one that was supposedly listed in the rules (single armageddon game?).

Of course, SB and head-to-head record could've been used as well and gotten the same result. But the final blitz games were exciting to watch.

[mjlef]

I am partly to blame about getting Komodo's hardware updated on the web site. We were able at he last minute to get access to a bigger, faster machine, but I should have gotten the TD the new specs. Sorry.

The 5 minus + 5 secs and 3 minute games were the only thing we could come up that seemed fair. Armageddon is just going to be to easy for the side with draw odds. I think everyone open to better suggestions, but we played a lot of games (45 min then 5 min then 3 min). We were fortunate that the operators were fast and did not lose the games on time.

It would be great to get a clear result in the original games, but as you note this was probably the strongest chess ever, and that will mean a lot of draws between the strong programs.

So ideas to make tie breaks better?

Mark

[Dan Cooper]

So ideas to make tie breaks better?

Mark

Arm wrestling between engine authors.