World Computer Chess Championship

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: World Computer Chess Championship

Post by jdart »

One reason to allow unlimited hardware is to enable programs to participate that require unusual hardware or can benefit from it.

Example: back pre-Crafty, Bob's program Cray Blitz competed, running on a Cray supercomputer. It didn't run on anything else.

More recently, Johnny competed running on a very large computer cluster (it has support for that).

AlphaZero of course is another example that uses custom hardware.

--Jon
leavenfish
Posts: 282
Joined: Mon Sep 02, 2013 8:23 am

Re: World Computer Chess Championship

Post by leavenfish »

jdart wrote:One reason to allow unlimited hardware is to enable programs to participate that require unusual hardware or can benefit from it.

Example: back pre-Crafty, Bob's program Cray Blitz competed, running on a Cray supercomputer. It didn't run on anything else.

More recently, Johnny competed running on a very large computer cluster (it has support for that).

AlphaZero of course is another example that uses custom hardware.

--Jon
Yes...but it's a bit like me having a race where the only similarity is that all are living creatures....one is a horse, one a mole, one an elephant, a human, a human with augments limbs (think South Africa's 'Bladerunner')....etc. Rather than say a race among similars - among horses or among dogs or among humans. Would a race between Usaine Bolt and a cheetah and an armadillo really mean anything in the end? The 'hardware' that powers each is rather different.

It is in the end a curiosity, not an actual test of the software itself against other software. Yet the software provider can claim 'it' is the winner...a 'paper title' one might say.
User avatar
Harvey Williamson
Posts: 2010
Joined: Sun May 25, 2008 11:12 pm
Location: Whitchurch. Shropshire, UK.
Full name: Harvey Williamson

Re: World Computer Chess Championship

Post by Harvey Williamson »

jdart wrote:One reason to allow unlimited hardware is to enable programs to participate that require unusual hardware or can benefit from it.

Example: back pre-Crafty, Bob's program Cray Blitz competed, running on a Cray supercomputer. It didn't run on anything else.

More recently, Johnny competed running on a very large computer cluster (it has support for that).

AlphaZero of course is another example that uses custom hardware.

--Jon
There are 2 tournaments. The WCCC that allows unlimited hardware and the WCSC which is played on a uniform platform.
Modern Times
Posts: 3546
Joined: Thu Jun 07, 2012 11:02 pm

Re: World Computer Chess Championship

Post by Modern Times »

jdart wrote:One reason to allow unlimited hardware is to enable programs to participate that require unusual hardware or can benefit from it.

Example: back pre-Crafty, Bob's program Cray Blitz competed, running on a Cray supercomputer. It didn't run on anything else.

More recently, Johnny competed running on a very large computer cluster (it has support for that).

AlphaZero of course is another example that uses custom hardware.

--Jon
Yes - and to me that unlimited hardware tournament is the most interesting.

And to have a level playing field there is the separate uniform hardware contest as Harvey says. So everyone is catered for.
mjlef
Posts: 1494
Joined: Thu Mar 30, 2006 2:08 pm

Re: World Computer Chess Championship

Post by mjlef »

Rebel wrote:
Vizvezdenec wrote:I don't really know how +200 elo engine can be a "derivative".
It's like... All engines are derivatives of Crafty, Fruit, Stockfish in some ways. If someone takes stockfish code and improves it by 200 elo, well, it's not stockfish anymore either way.
As you can read in the snippet of my previous post one reason the ICGA insists participants to have written their code from scratch is that programmers have complained Rybka had an unfair advantage by starting from Fruit, which BTW did not even happen. So no derivatives allowed, unfair competition.

It's crazy to see an organization to promote computer chess has become an obstacle to progress. The world has changed, everyone has moved on except for the ICGA that still wants to live in the previous century with outdated rules.
I see Ed has reformatted the same misleading information. I recommend anyone interested in seeing the data compiled by the investigation panel review it themselves her: http://icga.wikispaces.com/Rybka-Fruit%20Controversy. You will find links and summaries of the information all there.
mjlef
Posts: 1494
Joined: Thu Mar 30, 2006 2:08 pm

Re: World Computer Chess Championship

Post by mjlef »

Modern Times wrote:
mjlef wrote: Why would evidence of wrongdoing be covered up? TO me that would be like trying to cover up a crime.
Allegations of wrongoing, in your opinion, not proven by the relevant authorities. You are entitled to enforce your own rules in whatever way you like and to whatever evidential standards you see fit, in private but you are not a public protector. Everyone has basic human rights, and it seems that the ICGA have no understanding of that whatsoever.
Are you advocating covering the fact that a programmers stole from others? Really? I know of no "right to plagiarize". I do not see how anyone has a right to privacy because of taking things from others. The public has a right to know if something against the rules was done.
leavenfish
Posts: 282
Joined: Mon Sep 02, 2013 8:23 am

Re: World Computer Chess Championship

Post by leavenfish »

Modern Times wrote:
jdart wrote:One reason to allow unlimited hardware is to enable programs to participate that require unusual hardware or can benefit from it.

Example: back pre-Crafty, Bob's program Cray Blitz competed, running on a Cray supercomputer. It didn't run on anything else.

More recently, Johnny competed running on a very large computer cluster (it has support for that).

AlphaZero of course is another example that uses custom hardware.

--Jon
Yes - and to me that unlimited hardware tournament is the most interesting.

And to have a level playing field there is the separate uniform hardware contest as Harvey says. So everyone is catered for.
Very good then! And I agree, there is something perversely interesting about the differing hardware tourney.

Still, 'same platform'...with no Stockfish or Houdini can't be considered a true 'world champion'...just don't know why they do not show up.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: World Computer Chess Championship

Post by Rebel »

mjlef wrote:
Rebel wrote:
Vizvezdenec wrote:I don't really know how +200 elo engine can be a "derivative".
It's like... All engines are derivatives of Crafty, Fruit, Stockfish in some ways. If someone takes stockfish code and improves it by 200 elo, well, it's not stockfish anymore either way.
As you can read in the snippet of my previous post one reason the ICGA insists participants to have written their code from scratch is that programmers have complained Rybka had an unfair advantage by starting from Fruit, which BTW did not even happen. So no derivatives allowed, unfair competition.

It's crazy to see an organization to promote computer chess has become an obstacle to progress. The world has changed, everyone has moved on except for the ICGA that still wants to live in the previous century with outdated rules.
I see Ed has reformatted the same misleading information. I recommend anyone interested in seeing the data compiled by the investigation panel review it themselves her: http://icga.wikispaces.com/Rybka-Fruit%20Controversy. You will find links and summaries of the information all there.
You entirely missing the point I was making and in your reply only concentrate on the "which BTW" part which was added to avoid that folks get the impression I have changed my mind.

Therefore allow me to say it more blunt - by banning Rybka --- since the ICGA called it a derivative --- you have set a precedent that will hunt you until you come to your senses. Basically your organization insist that every participant is obliged to have his engine written from scratch.

In the meantime (and 7 years after) I expected something better but it is still the same old angry men gang that insists talented programmers to go through the same old shit (from scratch) because they had to.

You are a laugh to computer chess.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: World Computer Chess Championship

Post by Rebel »

Modern Times wrote:
mjlef wrote: Regarding your loan rejection, in the United States, anytime you apply for credit becomes a part of your credit record. This is because people who apply for a lot of credit often are bad credit risks. And since any loans you do take out become part of your credit report, it is not hard for a future lender to see you were rejected (although they would not know why).
A future lender yes, and there are strict controls as to who can access your credit record, and even then only with your permission. Certainly not available to the public at large.
mjlef wrote: I think rejecting an engine and not letting others know why is lack of transparency. All of this is up to the ICGA Board,
Not a lack of transparency at all, it is a matter of Privacy. Only if you think laws have been broken would you go further, and then you'd pass the information on to the appropriate authorities, not the world at large.
Give it up, it's still the same "name and shame them on the internet" mentality what's driving them.

While in the meantime Komodo (and many others) has taken more stuff from other sources than Vas ever did from Fruit.
pijl
Posts: 115
Joined: Mon Sep 17, 2012 8:59 pm

Re: World Computer Chess Championship

Post by pijl »

As I've participated in many different tournaments over time, and organized a few of them as well I like to put my 2 cents on my perspective. I'll try to structure them around a few topics.

Hardware
Anything goes in ICGA, although there is also uniform platform competitions. The main difference with TCEC is that the player decides on how to use the HW, including changing bios settings and putting anything on its disks that it can hold. With tournaments like TCEC it is the tournament director that sets the boundaries for the HW use (which also restricts the use of software features).
Although the participant is full in control of the use of the hardware, not everybody can get hold of nice HW for playing (and testing, which is probably as important). This adds some unfairness (or cost) as well.

Software
Again: anything goes (as long as you made it yourself of course). All software features can be used in ICGA, where important features are disabled in TCEC, like own book, learning, pondering etc.
ICGA allows to change SW setup or even SW between rounds to prepare against a specific opponent (or correct problems spotted in previous rounds), similar again as with other sports where the starting lineup of a team is often not the same in consequetive games. In TCEC you can only change the setup/software between tournament phases (unless there are severe stability issues).

Opening Book
Online tournament like TCEC usually restrict the use of opening books. I consider the opening book an integral part of the chess engine. Not being able to play with the own book and being forced in lines that the engine has trouble playing is not getting the maximum out of an engine.

Number of Rounds
Obviously, playing more rounds has the promise of better results, but more rounds take time. TCEC runs over several months, where the WCCC runs in a week. When compared to world championships in other sports (like football, where the champion plays 7 games, and half the participants are eliminated after only 3 games) a large number of rounds/games does not seem to be a precondition of awarding world titles.

Presence
Meeting other programmers in person is an important reason to join in-person tournaments. Not only the WCCC but also the CSVN and GSEI tournaments.
When the CSVN relaxed the rule on presence of programmers to attract stronger programs it decreased the fun I (and several other programmers) had in joining this tournament. This was the reason I started to organize my own programmer's tournaments.
For those that do not think this is important (and never been to such a tournament): Do you ever go to concerts? Buying the record, or streaming it from internet is cheaper and more convenient and gives you a broader choice in songs. Or is there a difference in experience after all?

Involvement
It is nice to see your program play in competitions, but for me TCEC is fundamentally different from any other basement tournament. Once the software is sent it is out of your hands. I'm participating to these type of tournaments as I still like to see it play and match it with other programs, but if I cannot I don't lose any sleep over it (unless it is because the Baron is playing and I want to see the game to the end).
With tournaments where you're playing yourself (also in online tournaments like CCT) the involvement is much higher as you are the only one responsible for setting up your program, choosing the configuration and adapting it if needed (e.g. plug holes in booklines, disable buggy features etc.). To use another analogy from football: It is the difference between watching on television and coaching a team.

Cost
Participating in a tournament where you need to be present (or available) costs money as well as time. So travel costs, hotel, meals, drinks etc. did cost me quite a bit of money as well as the days off from work. Participating at TCEC is for free.

Time
Participating in the WCCC/WCSC requires taking holidays from work and family (or at least dedicating a free weekend) to be able to play. The TCEC runs for months, but doesn't require that you watch. You can watch if you want, but if you don't that's fine as well. It doesn't require you to take time off.
For me this is what decides whether I play the WCCC this year or not. I plan to go to at least the WCSC (as that is shorter) but it depends a little on the tournament schedule as well.

Spectators
Here's where TCEC is the champion with superb online coverage. Even though improvements have been made by the ICGA and other tournaments by providing online coverage, this is still a point where improvements can be made. But also here there have been initiatives in the past that show how this can be done. I remember the WCCC in Pamplona where there was a live stream with comment (in Spanish though) by a leading chess journalist (don't remember his name but I'm sure someone else can help here). However, spectators that are going to the venue as well, will be able to talk to the programmer's present. In TCEC that's possible with the authors that are in the chat window.

I'm sure there are other points that can be added to this.
For me it is clear: Both formats have its merits and have a place in computer chess. Only cost and time restrict me from joining all tournaments.

Richard.