What the computer chess community needs to decide

[hgm]

I did not state my opinion on whether this is how it should be, I'm just saying this it is how it really IS and that is not an opinion.

That isn't tue a all. My programs have participated in ChessWar since 2006. I always participated with multiple engines, and never was a prize / trophy awarded to the winner. I participated with multiple engines to CCT and WCRCC several times. I don't think the winner there gets something either. I participated in ICT Leiden with two engines. (and so did Richard Pijl and Gian-Carlo that year). Shredder participated in two versions (themobilephone and he regular version) in Richard's Programmers Tour (also no award...). What you describe happens, but it is the exception, rather than the rule.

I have played in a lot of computer chess tournaments and in most of them there was some kind of prize that was awarded. Often a trophy but also I have received money. I did not give the money to my program, I kept it for myself! The money and/or trophy was awarded to ME, not my program. I think that is irrefutable evidence that this is how people think about these tournaments.

I see that merely as 'collecting debt'. If you count the fee the program owes you at a decent programmers wages, I am sure the prize doesn't even put a dent in it. Even if it is a $100,000 prize, as has sometimes been awarded.

Now, where opinion comes into the picture is whether you or I believe this is healthy. Do we believe this is how it should be? That would be an opinion and we might agree or disagree on this point. But it has nothing to do with how it actually IS.

When the computer chess world championships are held, it's strictly one program per author. If this is really a contest between programs (with no ego's involved) then it would make sense to allow people to bring as many different versions as they wish. Do you think that is likely?

Hard to say what is likely when others decide. It is certainly not impossible, as the numerous examples I quoted above show it is the exception. But a World Championship is supposed to be exceptional. But what is certain is that I do not participate in it because of this.

[Don]

I did not state my opinion on whether this is how it should be, I'm just saying this it is how it really IS and that is not an opinion.

That isn't tue a all.

It IS true. All you are saying is that YOU don't view it that way. But I am quite sure (and yes this part is an opinion) that even in these events you speak about, most observers and participants view it as a contest.

Do you really think the majority don't think of the programmers at all? I guess we cannot answer that without some kind of vote, so I will just agree to disagree on this one.

My programs have participated in ChessWar since 2006. I always participated with multiple engines, and never was a prize / trophy awarded to the winner. I participated with multiple engines to CCT and WCRCC several times. I don't think the winner there gets something either. I participated in ICT Leiden with two engines. (and so did Richard Pijl and Gian-Carlo that year). What you describe happens, but it is the exception, rather than the rule.

I have played in a lot of computer chess tournaments and in most of them there was some kind of prize that was awarded. Often a trophy but also I have received money. I did not give the money to my program, I kept it for myself! The money and/or trophy was awarded to ME, not my program. I think that is irrefutable evidence that this is how people think about these tournaments.

I see that merely as 'collecting debt'. If you count the fee the program owes you at a decent programmers wages, I am sure the prize doesn't even put a dent in it. Even if it is a $100,000 prize, as has sometimes been awarded.

Now, where opinion comes into the picture is whether you or I believe this is healthy. Do we believe this is how it should be? That would be an opinion and we might agree or disagree on this point. But it has nothing to do with how it actually IS.

When the computer chess world championships are held, it's strictly one program per author. If this is really a contest between programs (with no ego's involved) then it would make sense to allow people to bring as many different versions as they wish. Do you think that is likely?

Hard to say what is likely when others decide. It is certainly not impossible, as the numerous examples I quoted above show it is the exception. But a World Championship is supposed to be exceptional. But what is certain is that I do not participate in it because of this.

[Gian-Carlo Pascutto]

I participated in ICT Leiden with two engines. (and so did Richard Pijl and Gian-Carlo that year).

I did so only after my repeated protests that it was a very bad idea fell to deaf ears.

Participating with multiple engines increases the chance of winning. So it's a no-brainer to participate with as many engines as possible. But I only had 1 wife to help me

But again, it was a bad idea then, and it's still a bad idea now.

[Don]

I participated in ICT Leiden with two engines. (and so did Richard Pijl and Gian-Carlo that year).

I did so only after my repeated protests that it was a very bad idea fell to deaf ears.

Participating with multiple engines increases the chance of winning. So it's a no-brainer to participate with as many engines as possible. But I only had 1 wife to help me

But again, it was a bad idea then, and it's still a bad idea now.

I participated in a world championship with 2 engines many years ago. After the event was over and I thought more about it, I did not feel right about it. There were an odd number of contestants and my lesser entry had no real chance, but still it should not have been allowed to participate.

I think the organizer felt the same way I did initially, that it would have no real impact on the tournament but it is still wrong in principle.

Image a possible (although unlikely) scenario: My two programs finish first and second place. Do you think anyone would have said anything?

[hgm]

It IS true. All you are saying is that YOU don't view it that way.

Well, I kind of refuted all your arguments, (and you even tell now that the world championshipis no exception), so if you still want to continue believeing it, and admit that I can view it in a different way, it must be an opinion...

But I am quite sure (and yes this part is an opinion) that even in these events you speak about, most observers and participants view it as a contest.

Do you really think the majority don't think of the programmers at all? I guess we cannot answer that without some kind of vote, so I will just agree to disagree on this one.

[hgm]

Participating with multiple engines increases the chance of winning. So it's a no-brainer to participate with as many engines as possible. But I only had 1 wife to help me

I doubt that very much. You think you could come up with 3 or 4 engines that share no code, and each stand a chance of winning? Or even 3 or 4 engines that share code, but are so differently tuned that they would pass Don's similarity test as unrelated, and do not take a major hit in strength?

I challence you to try that, because it is really interesting: if many points on a multi-dimensional function have an approximately equal value, they are usually in the wing of a peak, and a much higher maximum lies in between.

[Don]

Participating with multiple engines increases the chance of winning. So it's a no-brainer to participate with as many engines as possible. But I only had 1 wife to help me

I doubt that very much.

I think you are either just messing with us, or you forgot to have your coffee this morning!

You think you could come up with 3 or 4 engines that share no code, and each stand a chance of winning? Or even 3 or 4 engines that share code, but are so differently tuned that they would pass Don's similarity test as unrelated, and do not take a major hit in strength?

I challence you to try that, because it is really interesting: if many points on a multi-dimensional function have an approximately equal value, they are usually in the wing of a peak, and a much higher maximum lies in between.

[Gian-Carlo Pascutto]

I doubt that very much. You think you could come up with 3 or 4 engines that share no code, and each stand a chance of winning? Or even 3 or 4 engines that share code, but are so differently tuned that they would pass Don's similarity test as unrelated, and do not take a major hit in strength?

Uh, yes. Well, about 2 I'm pretty sure. I didn't try 3 or 4 yet. Also depends on what you call a "major hit in strength" of course. Note that I don't need to finish 1 and 2: I just need to have the weaker engine strong enough that it can pick some points or half's of most of the other programs, and then play a massive blunder when it's playing me. That's enough to noticeably boost my chances for the end victory with the stronger engine.

I challence you to try that, because it is really interesting: if many points on a multi-dimensional function have an approximately equal value, they are usually in the wing of a peak, and a much higher maximum lies in between.

Or they're sitting on a big plane. I can change many, many things in my current engine that produce a very different search and eval and don't have it move an inch in strength. I find this very counter-intuitive, but it is the situation...

[Don]

Participating with multiple engines increases the chance of winning. So it's a no-brainer to participate with as many engines as possible. But I only had 1 wife to help me

I doubt that very much. You think you could come up with 3 or 4 engines that share no code, and each stand a chance of winning?

This isn't about whether you stand a good chance, it's about increasing your chances period. If there were 100 entries and I had the weakest program, my chances are slim. But if I had the 5 weakest programs my chances have enormously increased - although they still remain slim.

It's like smoking a cigarette, smoking 1 is not likely to kill you. Smoking 10 is also not likely to kill you, but it's a lot more likely to kill you than smoking 1. No single cigarette is likely to cause any real harm.

Come on, I know you are far more skilled in mathematics than I am, I don't really understand what it is you are trying to refute here. You know this as well as I do.

Or even 3 or 4 engines that share code, but are so differently tuned that they would pass Don's similarity test as unrelated, and do not take a major hit in strength?

Why are you putting all those constraints on this? Why do they have to pass any tests? The argument is that we don't care about authorship. I want to enter 3 program with 3 different books for instance to see which book does best - and they are clearly 3 different entities. Or perhaps I want to try 3 different versions with slightly different mobility values for the bishop.

If the programs are far weaker then your odds do not go up as much, but it still goes up and that's the point.

A simple model to understand this is that all programs have equal chances with the same strength and the tournament is constructed such that only 1 program can win.

Let's assume there are 24 programs and yours is one of them. you have probability 1/24 to win our about 0.0417 if 2 of those 24 programs was yours your odds go to 2/24 or 0.0833. the more programs you have, the better your odds.

I challence you to try that, because it is really interesting: if many points on a multi-dimensional function have an approximately equal value, they are usually in the wing of a peak, and a much higher maximum lies in between.

At the Hong Kong tournament the Deep Blue team (Campbell) confided to me that they estimated their chances of winning to be about 50%. This was based on their belief that Deep Blue was far stronger than everyone else, but that they had to outperform many other programs. Assuming that their calculation was correct, what if ALL the other program belonged to a single author? You would be in a position where "your" odds of winning the tournament were as good as the Deep Blue authors chances.

[mjlef]

There is a growing community of programmers that like the idea of public sources they can modify and improve. But I do not think we want to see a bunch of nearly identical programs competing against each other. As Don states, since tournaments cannot have a large number of rounds, multiple entries of nearly identical programs greatly increases the chance that one of the nearly identical programs will win. So, how do we deal with the open source issue?

I propose each program must reveal the code base used in their program (either original, or based on program X). if more than one program based on X enters the tournament, a pre-tournament must be held with a few games between each of the programs based on X. Only the winner of this pre-tournament would be allowed to compete in the final tournament. Also, derivatives would have to follow the GPL or other licence conditions, of course, which might disallow this.

This does not deal with the issue of clones who hide the origin. And it would probably annoy the original authors if their program gets surpassed by a derivative. If this is the set rule, it might make people think twice before releasing code. But it does have the nice feature of increasing the likelihood that the strongest derivatives will play, and seems like a fair way to proceed.

Mark