A compromise idea for the WCCC hardware limit

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: A compromise idea for the WCCC hardware limit

Post by bob »

Kirill Kryukov wrote:
Mike S. wrote:Unlike during the 1980s/90s, when there were two tournaments separated from each other, maybe world champion titles in two categories could be awarded in one event, alltogether.

:arrow: (1.) The absolute Computer Chess World Champion
(everything unlimited)

:arrow: (2.) The Uniform Platform Computer Chess World Champion
(consumer-orientated hardware limit; other limits like for opening books are debateable IMO but I won't stress that.)

Some will say, what do we need (2) for, or I am only interested in (1). Others will say, (1) is meaningless for me, or with the equipment I can afford I have no chance in (1). So, this proposal is only for people who can accept, and want to handle compromises. A compromise can provide the chance to include all parties of interest, unlike when an extreme position wins and others are excluded (for whatever reason or in whatever role).

I think it can be done if up to two entries are allowed, per participant. But in that case, one entry would have to play for (2.) with limited hardware. In case of only one entry, he should have the free choice of category. Maybe a long swiss tournament could be done in a way that whenever possible, the category of the opponent is switched, round by round (like the switch of colors).

The idea of uniform platform raises the problem - which maybe also is a chance though - to find a sponsor for the required number of identical computers, on site.

Alternatively, the particpants would need to bring in "very similar" hardware for category (2). But I think that is only the second best choice, because even with adjustments, they can probably not be standardized in the sense of uniform platform. - Or the requirement for / definition of "consumer-orientated hardware limit" is regulated in a way that it allows a certain bandwidth. Limiting the number of cores only, seems very general.

Also, for now - but probably not for the near future - I think 4 cores would be a better limit, because as explained by Jens, there are huge cost and performance differences within an 8 core limit, still. We have had similar thoughts here, because the same topic was being discussed at the Rybka forum some time ago too, where I had suggested u.p. with quads.

As for the verification of how many cores are used: This just cannot be a problem in a place where computer experts gather together. For example, Microsoft's Process Explorer doesn't even need to be installed, so the tournament director could have it on an USB stick. So nobody can hack it to display a faked CPU load (in case if lovers of conspiracy theories had such concerns :mrgreen:). There are also free CPU identification tools, like CPU-Z.

http://technet.microsoft.com/en-us/sysi ... 96653.aspx

(I am almost sure that such ready-to-run tools also exist for Linux and Mac, but I don't know it.)

Anyway, I am aware myself that my proposal is not very realistic, but it should show that a compromise is possible, at least as something "theoretically imaginable"...
This "compromise" is basically identical to just event 1 existing, because no one will care about event 2, unless it has a very attractive prize (which I doubt will happen).

We already know how various engines stand with "consumer-oriented hardware limit", there is enough data accumulated and published online.

"Consumer-oriented" event may at first sound like a good idea, but to just limit hardware is only one step towards the consumer. For an event to be really consumer-oriented, only publicly available engines should be allowed to compete. With publicly available books as well, and on available common hardware.

Then of course there is question how do the consumers actually use a chess engine. If most of the consumers use chess engines for analysis, rather than for playing with them (which I believe to be true), then using a shared shallow opening book would give a better picture about how well various engines are suited to this purpose.

Then there is a question: what conclusion can a consumer draw from a result of such an event? (event 2). Obviously the statistical significance is too low for the result to be of any use for consumer's decision-making. Although world title may be useful for marketing, it is actually misleading (as most people don't know how to interpret tournament results), and therefore is against the interst of the consumer.

So the whole idea of the consumer-oriented event is flawed from about every angle.

Plus, with ongoing testing projects, providing thousands of games under a variety of testing conditions, the consumer already knows all he needs to know about the comparative strength of chess programs.

What consumer in fact wants from a WCCC is the battle of the most advanced computer chess players that could be created. Consumer wants to be thrilled. Uniform Platform even would not thrill anyone. No one will even watch it, other than true fanatics like us.

Clearly the "Uniform Platform", or "Limited Platform" is not interesting for the consumer. As evident from this and other forums, it is also not interesting for the absolute majority of programmers. The natural question is then: who needs a hardware limit?". Probably only ICGA needs it, lobbied by those commercial programmers (are there more than 1?) who already possess an 8-core hardware.

The reason why those commercial programmer(s) would want such limit is not only to weaken the competitors (eg Rybka). I think they foresee that with hardware limit there simply will be much less of competition at all. Because many authors will decide to ignore the event. So those commercial programmers have double benefit. They will play on their fine-tuned 8-core machine, with a few remaining opponents. And with a good probability they will then display "World Champion 2009" in their ads.

Question is: How the ICGA could fall for this, and if there are other factors (unknown to us) that influenced their decision.

I directly asked David this question, via email. If / when I get an answer, I will explain it here. I personally believe it is the result of commercial programs whose authors are unwilling to work on parallel search issues, and rather than getting their feet wet and their hands dirty learning how to do it effectively, they prefer to prevent anyone else from using these approaches instead...

More when / if I get a response.


Note that form the ICGA point of view the decision is totally stupid, as it destroys the world title, and without the world title, who needs the ICGA? May be ICGA will have more success organizing Hex of Connect 6 tournaments?

Best,
Kirill
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: A compromise idea for the WCCC hardware limit

Post by diep »

George Tsavdaris wrote:
Thomas Mayer wrote: By the way, in your system: If Hydra plays on a quadcore, how should it be handled ?
No need to wonder. The answer is simple. In his system Hydra has no place. It should not participate in the WCCC. He said that clearly in another post.
WCCC is no place for Hydra with the big hardware.
I wonder if he actually believes what he writes. :roll:
Without programmers the WCCC will die anyway and with such limitations you kick the programmers away.
Yes but the spectators will be happy after all with their fair conditions for the tournament. It doesn't matter if tournament will not take place as long as conditions are fair. :D
Which spectators?

In the last WMCCC 2000 i saw only 1 spectator, and his name was Richard Lang (nice to speak to him).
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: A compromise idea for the WCCC hardware limit

Post by bob »

Harvey Williamson wrote:
bob wrote:
Harvey Williamson wrote:
George Tsavdaris wrote:
Harvey Williamson wrote: Do you think a sponsor would have any interest in the even if they read Bob's statement above?
Bob's statement is for the current WCCC and the way it's being held until now. (Programmers pay those high prices to enter and spectators contribution is near zero.)

If situation changes and spectators become the main contributor of the event, perhaps Bob's statement becomes that WCCC is for programmers and spectators.
Bob pays nothing to enter as his program is not commercial.
Sorry, but that's wrong. I pay travel, room and board. I have to take two weeks off from work. I have to arrange hardware. It is not free...
I pay that + 500 Euros and take the time off work. We intend to take part you have made it clear for some time that you won't so you will not have to make this sacrifice, one I am happy to make.
I will tell you exactly what I told David Levy many times over recent days. When the WMCCC was first started, it was commercial-only, and commercial chess authors are not exactly "taking time off from work" to go spend a week or two at some exotic location to play in a tournament. The WCCC was always a short-duration event to attract interest and participation from those that were not doing chess stuff full-time.

there is a _big_ difference. Do you believe it reasonable that I miss two weeks of class, with students paying tuition to take my classes, so that I can attend a computer chess event that won't help _their_ education in any possible way???

Neither do I.

BTW, I looked at the ICGA web site. No Hiarcs in 2007. Hmmm no Hiarcs in 2006. Damn... no Hiarcs in 2005. So are you _really_ a regular competitor?
User avatar
Harvey Williamson
Posts: 2026
Joined: Sun May 25, 2008 11:12 pm
Location: Whitchurch. Shropshire, UK.
Full name: Harvey Williamson

Re: A compromise idea for the WCCC hardware limit

Post by Harvey Williamson »

diep wrote:
George Tsavdaris wrote:
Thomas Mayer wrote: By the way, in your system: If Hydra plays on a quadcore, how should it be handled ?
No need to wonder. The answer is simple. In his system Hydra has no place. It should not participate in the WCCC. He said that clearly in another post.
WCCC is no place for Hydra with the big hardware.
I wonder if he actually believes what he writes. :roll:
Without programmers the WCCC will die anyway and with such limitations you kick the programmers away.
Yes but the spectators will be happy after all with their fair conditions for the tournament. It doesn't matter if tournament will not take place as long as conditions are fair. :D
Which spectators?

In the last WMCCC 2000 i saw only 1 spectator, and his name was Richard Lang (nice to speak to him).
Hans and I managed to broadcast all our games, from Beijing, to watching 1000's on Playchess.
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: A compromise idea for the WCCC hardware limit

Post by diep »

bob wrote:
Harvey Williamson wrote:
chessfurby wrote:**I also spend hours every day working on Hiarcs. i do not get paid for my work. However I value the fans, users and spectators. A world championship is about many things including of course the programmers


Same is true for me. Spending hours testing Sjeng etcetc. I am in it for the fun. The part where it turns "no fun at all" is that we spent quite some time setting up and testing a cluster which GCP was successfully able to do. Quite a lot of work went into writin, testing etcetc.

Allofasudden it is no longer allowed?!?!? Oh yeah .. sure it must be in the audiences best interest if the skills of some participants are severely hindered and artificially limited, just because others do not (or are skillwise not able to) do the work that comes with implementing a cluster/ smp/ GPU/ fpga version of his or her engine....

It is also not fun if- on relative short notice- you are being told "Ey listen all the time that u put into those projects is... err... NIL".
How much notice do you want? GCP is welcome to borrow my spare 8 core as he has done before. However I know you spend a lot of time on Sjeng and the point I am making is that Bob's comment is just so strange about a WORLD championship.
No, you are just not paying attention. The competition itself, is solely for the programmers. Write your best. Find the best hardware. Test the best way you can. Develop the best book. Then show up with that stuff and compete against the best that everyone else has.

What you want is to turn this into a completely uninteresting and irrelevant tournament, which will always be subject to the criticism "but the 16 core fritz in the playchess server could thump _any_ of the programs in the WCCC including the winner" and you would have absolutely no defense to that claim, and it would render the WCCC completely irrelevant.

Makes no sense to me. Yes a _few_ non-programmers are interested in the results. Those are in the thousands, not millions. Sponsors could care squat about providing a lot of hardware that might be seen by a few hundred at most, and then forgotten about because the hardware was so run-of-the-mill. But ask them to showcase their latest big server or cluster, and things are different, because this produces a "buzz" far beyond just chess. Cray Blitz used to have a big "fan club" at the super-computer conferences where many ACM events were held. Guess why? Not because of "chess" because most of these people didn't even play chess. But they were interested in supercomputers and wanted to see their "favorite brand" win another tournament and show the superiority (an impression to be sure, but still there) of the Cray brand over all the others.

You want to render the WCCC completely irrelevant, which is fine by me since I consider the CCT events to be more prestigious anyway since there are far more entrants and still with sophisticated hardware platforms.

BTW, for "notice" a _year_ should be the minimum. I am working on the design for a "cluster Crafty." Which will probably be ready in time for the WCCC if we choose to enter. But apparently that won't even be possible and all the effort expended so far is for nothing. Better would be two-three years if it is such a major and radical departure from the past 40 years of computer chess history.
You know, i shall only quote Stefan MK:
"if Hydra joins somewhere i can get anything done from a sponsor".

Idemdito if some superclusters join, the tournament has more prestige with a sponsor then.

As simple as that.

Many world champs i have tried to convince programmers to join. Very succesful was 2005. I managed to interest the guy who later on won the event :)

Most programmers are not interested: "because my program is not strong enough yet". Getting parallel is just 1 step to get stronger, just like pondering is and just like a better book is.

It is true that usually US programs have fastest hardware. Exception was 2003 world champs. diep was the only supercomputer there. Parallellizing Diep in that 1.5 years between 2002 world champs and 2003 meant i didn't do anything to evaluation function. Whereas in 2002 that thing kicked butt, by 2003 it was outdated (once again). Progress goes so fast.

Suppose you organize a world championship where only evaluations are allowed that consider less than 100 patterns.

That's same thing like parallel hardware.

More processors cause more hashtable problems, the more selective you search, the bigger the problems.

I'd say that parallel machines are mighty interesting for the progress of the field to investigate.

Now computer-go guys run at open hardware, whereas in computer-go the hardware matters more than in chess, and computer chess is not using open hardware.

See how silly that is?

Computer-go which is somewhere in the 70s, in which days hardware was a major advantage, there hardware is total open.

In computer chess it is not now so suddenly.

The explanation given is not even political acceptable (and oh boy we've seen past years a lot of weirdo explanations there for doing the most violent thing that mankind can do to itself).

Vincent
Marc Lacrosse
Posts: 511
Joined: Wed Mar 08, 2006 10:05 pm

Re: A compromise idea for the WCCC hardware limit

Post by Marc Lacrosse »

The competition itself, is solely for the programmers. Write your best. Find the best hardware. Test the best way you can. Develop the best book. Then show up with that stuff and compete against the best that everyone else has.
This is the one and only way for a Wch computer chess tournament to go.

I was involved in the 2005 Wch tournament with Fruit.
Fruit played with the miserable local HW and there were quite a few opponents with terrific superior hardwares there.
We began to prepare eight _days_ before the competition was to begin.
It was a wonderful experience for Fabien, for myself and for all those who participated in any way in our team.
This remains true although we did not win.

World comp chess tournament has to be the best on the best for the best at the time.

I could not even imagine anything else!

Marc
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: A compromise idea for the WCCC hardware limit

Post by geots »

I cant see where this argument matters at all anyway. Just as Bob Hyatt stated earlier quite a few days ago, unequal hardware or equal hardware- it really isnt going to matter in the end. Because you will still have too few games played to really determine much of anything- actually nothing- as far as strength and ratings are concerned. If you want the truth about comparable engine strength- go to CCRL or CEGT, as Bob suggested. I personally would like to see Hydra- or even Deep Blue (a wish) if ever remantled- play Rybka, Hiarcs, Shredder, Naum, Crafty etc.- which would not be possible under = hardware.

Best,
CThinker
Posts: 388
Joined: Wed Mar 08, 2006 10:08 pm

Re: A compromise idea for the WCCC hardware limit

Post by CThinker »

geots wrote:I cant see where this argument matters at all anyway. Just as Bob Hyatt stated earlier quite a few days ago, unequal hardware or equal hardware- it really isnt going to matter in the end. Because you will still have too few games played to really determine much of anything- actually nothing- as far as strength and ratings are concerned. If you want the truth about comparable engine strength- go to CCRL or CEGT, as Bob suggested. I personally would like to see Hydra- or even Deep Blue (a wish) if ever remantled- play Rybka, Hiarcs, Shredder, Naum, Crafty etc.- which would not be possible under = hardware.

Best,
Yup. CCRL and CEGT are now the best source of equal hardware tournament.

The last ACCA tournament just had the best open hardware tournament.

Which leads to the question - What is the WCCC for?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: A compromise idea for the WCCC hardware limit

Post by bob »

CThinker wrote:
geots wrote:I cant see where this argument matters at all anyway. Just as Bob Hyatt stated earlier quite a few days ago, unequal hardware or equal hardware- it really isnt going to matter in the end. Because you will still have too few games played to really determine much of anything- actually nothing- as far as strength and ratings are concerned. If you want the truth about comparable engine strength- go to CCRL or CEGT, as Bob suggested. I personally would like to see Hydra- or even Deep Blue (a wish) if ever remantled- play Rybka, Hiarcs, Shredder, Naum, Crafty etc.- which would not be possible under = hardware.

Best,
Yup. CCRL and CEGT are now the best source of equal hardware tournament.

The last ACCA tournament just had the best open hardware tournament.

Which leads to the question - What is the WCCC for?
It's primary function, according to the original charter of the ICCA, was to foster computer chess development around the world. Apparently it is being relegated to a pure fund-raising enterprise, but with an acute lack of interest, it is going to be more and more difficult to use it for that purpose. And without the WCCC type events, I am not sure the ICGA journal can survive since that is a major source of revenue that supports the journal.

Let's see what happens, since it appears to be a "done deal". If David is right, the WCCC will have far better participation than the upcoming CCT tournament which will still allow open hardware...
User avatar
Harvey Williamson
Posts: 2026
Joined: Sun May 25, 2008 11:12 pm
Location: Whitchurch. Shropshire, UK.
Full name: Harvey Williamson

Re: A compromise idea for the WCCC hardware limit

Post by Harvey Williamson »

bob wrote:
CThinker wrote:
geots wrote:I cant see where this argument matters at all anyway. Just as Bob Hyatt stated earlier quite a few days ago, unequal hardware or equal hardware- it really isnt going to matter in the end. Because you will still have too few games played to really determine much of anything- actually nothing- as far as strength and ratings are concerned. If you want the truth about comparable engine strength- go to CCRL or CEGT, as Bob suggested. I personally would like to see Hydra- or even Deep Blue (a wish) if ever remantled- play Rybka, Hiarcs, Shredder, Naum, Crafty etc.- which would not be possible under = hardware.

Best,
Yup. CCRL and CEGT are now the best source of equal hardware tournament.

The last ACCA tournament just had the best open hardware tournament.

Which leads to the question - What is the WCCC for?
It's primary function, according to the original charter of the ICCA, was to foster computer chess development around the world. Apparently it is being relegated to a pure fund-raising enterprise, but with an acute lack of interest, it is going to be more and more difficult to use it for that purpose. And without the WCCC type events, I am not sure the ICGA journal can survive since that is a major source of revenue that supports the journal.

Let's see what happens, since it appears to be a "done deal". If David is right, the WCCC will have far better participation than the upcoming CCT tournament which will still allow open hardware...
CCT is a good tournament but would there be a great turnout if everybody had to travel to the States for it? I wouldn't mind the trip :)