What the computer chess community needs to decide

[hgm]

I think the time people would marvel at the mere fact a computer could play an intellectual game like Chess are well behind us. Just as the times where people marvelled that you could drive a 'horseless carriage' over the road and actually reach the finish are long gone. Now the common reaction is more likely to be: "You beat the World Champion? Oh, but you were using a computer, so what?". "You did the marathon??? Oh, on a motor cycle. Shame on you..."

Common folkes won't realize that Go is a tad harder than Chess, especially in this part of the World. But for those looking for appreciation by human players, Shogi and Go could still be viable alternatives if you don't mind the only appreciation will get from the far East. (But these are emerging markets, after all... )

Otherwise, it is completely immaterial what game you pick IMO. So you might as well pick Spartan Chess, which is what I did (this year). And I don't think any of the participants regrets it. Even poor Catalyst, who finished last, but was improved so much during the tourney that it would not finish last on the next occasion.

[Don]

Hi Don,

A couple of things. First, concerning Rybka 4 being a version of Fruit "by transitivity", couldn't one simply say that this is no longer true IF Rybka 4 no longer has any Fruit code? In this case, I would think that any remaining "essence of Fruit" would simply be "copying ideas" instead of "copying code", and this is pretty much universally accepted as okay.

Second, you say the following:

He probably blundered by leaving some small bits unchanged, such as the UCI interface parts because presumably that is still identifiable.

Is this "still identifiable" in Rybka 3 and 4, or are you referring here to Rybka 1.0 Beta?

It's my understanding that Rybka 3 still has some non-engine parts of Fruit - but don't quote me on that. I have not looked myself as that is a lot of work.

Personally I would accept Rybka if it could be shown there is no Fruit code in the engine portion, but that leaves GPL issues which I'm trying to consider as separate. I am trying to focus on what we should consider acceptable (legal issues aside.) Is it reasonable to start from an existing code base and make heavy changes? If so, how heavy? What if it's not copy and paste but a more or less direct translation that is hand coded? How can we prove that ANY program is not based on some ancient ancestor? Is there a point at which heavy modifications transforms a program into an original? It's complicated by the fact that you can start from any point and end up at any point - every program can be transformed into any other program with a fixed number of changes.

So that only thing to me that makes sense is that we have to consider how different one program is from another. But even that is a messy and imprecise art. Of course we can deal with what is more obvious, which is basically how it's being handled now.

I think the real issue is who gets credit for the work anyone has done. There is a general principle in computer chess (from a competitive point of view) that you cannot be represented by more than one program in any event. For example I could take Komodo and make 10 different programs from it, give them completely different names and get 10 new email addresses and we could have 10 new programs by "new" authors and submit them for testing of CCRL and the other rating organizations. Then I could by emal get 10 different volunteers to take my program to tournaments. I could easily make enough changes to each version that it was not completely obvious what is going on - it would only be obvious when people take a hard look at them.

[paulo]

Wow, amazing statement IMHO. If you don't look at the state-of-the-art engines how can you improve your CC knowledge to apply to your engines?
Can you imagine, for example, a surgeon saying something similar, that he never looked at all to recent developed practices and techniques?

I don't have to look at actual source code to learn about latest techniques. They are discussed in this forum (in another section, to be sure!). My engines use null-move pruning, because it was explained here in the forum how this search strategy works, and that it was very benificial. Coding I can do myself, as well as the testing to confirm that it indeed was a big help. My engines use LMR, and I did not have to study the source code of Glauring to know how that worked. It wasalso explained here.

I must admit I did look at the code for the evaluation function of Elephant Eye, in particular the King Safety term. But this was more to get an impression what should be valued and what avoided in keeping your King safe in Xiangqi (a variant for which I had no feeling at all, as I have never played it myself). The King Safety term I ended up with in HaQiKi D looked completely different from that in Elephant Eye. But it does recognize some of the same patterns as Elephant Eye, like the "hollow threat" and "central threat", simply because these are elementary strategic patterns of that game. But I would never have known it, without playing, with virtually no English texts available on its strategy (and none on-line and free), without looking at the Elephant Eye evaluation. It was not so much for getting the state-of-the-art of computer Xiangqi, but more filling in my total lack of knowledge on basic game strategy.

I got it, thanks!

[jdart]

That's pretty much what I said.

For a long time the commercial closed-source programs were a long way ahead of the amateur and open-source programs. They still probably have some unrevealed secrets. I'd still like to see what is inside Fritz or Shredder. But now the open source engines have caught up and there is much less of a knowledge gap.

This has had a number of effects. It has already spawned a group of amateurs who seem to like modifying things more than creating them. It has made it more readily possible to produce a strong commercial engine that is competitive. It may ultimately reduce revenue for commercial chess engines, although there were never a lot of those that made money, and IMO there is a still a market for something that is supported, updated, and enhanced with UIs and other features. It's also generating a lot of controversy about legitimate vs. illegitimate code use, and tournament participation. There is no easy way to resolve that IMO.

And there is no going back to the previous state of things.

[hgm]

There is a general principle in computer chess (from a competitive point of view) that you cannot be represented by more than one program in any event.

This is not really a general principle. In ChessWar, 5 of my engines participate (Joker, micro-Max 4.8, micro-Max 1.6, Usurpator II and NEG.) It depends on if you consider it an event for athors or an event for engines. Events for engines are more interesting for me, and I hardly ever would register for any other.

Of course there is the problem of 'obvious cheating', such as you describe. No one would allow a multitude of infinitesimally different engines in one tournament. Not by the same author (who could easily produce such a multiude in the way you describe), but also not from different authors (rewrites of Fruit that all still have the same eval, but differening in speed by a few percent becuse of the different programming techniques used). As a tournament organizer I really would not care much if they were from the same or from a different author.

Note that this is quite different from rating lists: these usually do contain many versions of the same engine, and can even continue to test obsolete versions to determine their rating more precisely (e.g. for use as a reference point on their ratingscale).

So if I was organizing a serious tournament for engines, I would subect all engines to a similarity-detection test like you designed, and refuse entries that are too close in behavior to other engines. Which engine I would allow in case of a 'collission' would depend on the purpose I had for the tourney. Many models are possible here, like oldest release date, oldest author, first to subscribe, strongest engine, smallest number of entries, etc. If Rybka is sufficiently different from Fruit I would not care, and thus would not have to worry over, if one is a derivative of the other.

But that is of course because I do not care who gets the credit for the engine. It would be an event for engines, after all, and not for authors. So as far as I am concerned the creditstops at the engine. The authors can fight out amongst themselves who stole what from whom. Why would I want to get involved into that as a tournament organizer? I am not the police, and don't see it as my vocation to cure all wrongs in the World. Not even in the computer-Chess World.

[michiguel]

You are absolutely wrong. This is what really irritates me: The current tide of cloning is making everybody believe that everybody is the same. No. There are engines that started from scratch, being typed with the very hands of the author

int main (... etc.


If you are going to apply that criteria, Houdini will be one of the first to be out. You chose a wrong example here.

Miguel

No offense but I'm right. If you read my post carefully I didn't say authors always copied code. They at least translated some things, as Dr. Hyatt explained. No current author wrote a chess engine from scratch without the aide of learning from some other source. This is a fact.

No, it is not a fact, and you are wrong. There are authors that wrote their engine without studying others. You are making a blanket statement without knowing them all, and of course, you don't know them all.

I disagree. As it wasn't shown what is STATISTICALLY SIGNIFICANT ENOUGH. Prove that Houdini 1.5a plays statistically significantly similar to another engine. I'll let you choose a reasonable definition of statistically significant.

Yes, it is statistically significant more similar to some Ivanhoes more than any other engine out there. I has been shown already here with data from Kai and Adam. I ran for them the jackknife analysis and there is no doubt. I may accept different interpretations, but not to ignore the data.

Miguel

There are no current active authors who wrote engines without studying others. To believe so is asinine. If you think you are right prove me wrong (impossible to do).

My engine, Gaviota. Have you ever asked me? no. Have you even bother to look it up? it is in Gaviota's website acknowledgments. Not knowing that I am an author is truly understandable, but you can always imagine it.

HGM's already answered, which was already easy to predict. Do you think he came up with Micromax studying other people's sources? That is already two, so I do not have to go on.

However, your statement that proving you wrong is impossible surprises me, because it is quite easy to do. I need only one example. On the other hand, to prove your statement, you need to explore all the engines, which would be a bit more difficult task.

Saying they are is not bringing into existence and saying what you did doesn't make you right. You must SHOW here (this thread) and have your peers judge and come to their own conclusions. Just post a link.

Must?
This has been extensively discussed with peers. If you want me to provide you a link, the proper procedure is to introduce the word "please" somewhere in the sentence. Otherwise, without that magic word, the goddess Aergia cannot come to the rescue.

Miguel

[Don]

There is a general principle in computer chess (from a competitive point of view) that you cannot be represented by more than one program in any event.

This is not really a general principle. In ChessWar, 5 of my engines participate (Joker, micro-Max 4.8, micro-Max 1.6, Usurpator II and NEG.) It depends on if you consider it an event for athors or an event for engines. Events for engines are more interesting for me, and I hardly ever would register for any other.

Of course there is the problem of 'obvious cheating', such as you describe. No one would allow a multitude of infinitesimally different engines in one tournament. Not by the same author (who could easily produce such a multiude in the way you describe), but also not from different authors (rewrites of Fruit that all still have the same eval, but differening in speed by a few percent becuse of the different programming techniques used). As a tournament organizer I really would not care much if they were from the same or from a different author.

Note that this is quite different from rating lists: these usually do contain many versions of the same engine, and can even continue to test obsolete versions to determine their rating more precisely (e.g. for use as a reference point on their ratingscale).

So if I was organizing a serious tournament for engines, I would subect all engines to a similarity-detection test like you designed, and refuse entries that are too close in behavior to other engines. Which engine I would allow in case of a 'collission' would depend on the purpose I had for the tourney. Many models are possible here, like oldest release date, oldest author, first to subscribe, strongest engine, smallest number of entries, etc. If Rybka is sufficiently different from Fruit I would not care, and thus would not have to worry over, if one is a derivative of the other.

But that is of course because I do not care who gets the credit for the engine. It would be an event for engines, after all, and not for authors. So as far as I am concerned the creditstops at the engine. The authors can fight out amongst themselves who stole what from whom. Why would I want to get involved into that as a tournament organizer? I am not the police, and don't see it as my vocation to cure all wrongs in the World. Not even in the computer-Chess World.

If a tournament is considered a "contest" or competition, it is between authors. Otherwise it is an "experiment." or study.

To an outside observer who doesn't care anything about the process it could be seen as kind of abstract contest between non-human entities, but that's not really what it is.

There is a kind of tournament were a single person organizes the competition and plays matches and publishes the results, in some cases on-line. Those could possibly be thought of non-contests (or contests between programs only), but I think even those are real contests. Otherwise, why not have several versions of the same program? This is rarely allowed from what I see.

I think this is really what is at the heart of the debate however. If all we really care about is the program and not the people behind it, then by all means, anything goes. But in such a case I would suggest that the concept of authorship goes against this principle - because it doesn't really matter. We should all be given the same credit for the whatever the strongest program happens to be, because we are all in some abstract sense part of the process and any of us could make a copy and put our own names on it. So the very concept of authorship should be considered a kind of pompous vanity. We should not recognize the accomplishments of others.

This sounds strangely enough like the philosophy behind Ippolito doesn't it? They believe in competition but in their case it is not based on the individual, but on nationalistic bigotry.

[hgm]

I think this is just an opinion, and of course you have the right to have your own opinion. But I don't share it. I see no logical need why a contest would imply it is between authors. I consider my engines like my children. And I would consider it very unfair when the organizers of a Chess tournament would say to my second born: "sorry, we don't want you, because your brother already plays in this championship, and we allow only one sibling per family". Or that they say to my eldest: "sorry, we already have registered your younger brother, because he is stronger". When I was in highschool, both my brother and me were allowed to participate in the same championship.

That doesn't mean I can't feel proud if my children accomplish something great. Part of the glory of children rubs off on their parents. If anything, it proves they are good breeding stock.

[frcha]

You are absolutely wrong. This is what really irritates me: The current tide of cloning is making everybody believe that everybody is the same. No. There are engines that started from scratch, being typed with the very hands of the author

int main (... etc.


If you are going to apply that criteria, Houdini will be one of the first to be out. You chose a wrong example here.

Miguel

No offense but I'm right. If you read my post carefully I didn't say authors always copied code. They at least translated some things, as Dr. Hyatt explained. No current author wrote a chess engine from scratch without the aide of learning from some other source. This is a fact.

No, it is not a fact, and you are wrong. There are authors that wrote their engine without studying others. You are making a blanket statement without knowing them all, and of course, you don't know them all.

I disagree. As it wasn't shown what is STATISTICALLY SIGNIFICANT ENOUGH. Prove that Houdini 1.5a plays statistically significantly similar to another engine. I'll let you choose a reasonable definition of statistically significant.

Yes, it is statistically significant more similar to some Ivanhoes more than any other engine out there. I has been shown already here with data from Kai and Adam. I ran for them the jackknife analysis and there is no doubt. I may accept different interpretations, but not to ignore the data.

Miguel

There are no current active authors who wrote engines without studying others. To believe so is asinine. If you think you are right prove me wrong (impossible to do).

My engine, Gaviota. Have you ever asked me? no. Have you even bother to look it up? it is in Gaviota's website acknowledgments. Not knowing that I am an author is truly understandable, but you can always imagine it.

HGM's already answered, which was already easy to predict. Do you think he came up with Micromax studying other people's sources? That is already two, so I do not have to go on.

However, your statement that proving you wrong is impossible surprises me, because it is quite easy to do. I need only one example. On the other hand, to prove your statement, you need to explore all the engines, which would be a bit more difficult task.

Saying they are is not bringing into existence and saying what you did doesn't make you right. You must SHOW here (this thread) and have your peers judge and come to their own conclusions. Just post a link.

Must?
This has been extensively discussed with peers. If you want me to provide you a link, the proper procedure is to introduce the word "please" somewhere in the sentence. Otherwise, without that magic word, the goddess Aergia cannot come to the rescue.

Miguel

I think at this point we should have established that good engine authors do not need to copy /burrow from other engines to create their own.

Also that Good Programmers can write programs from scratch while not so good ones need to copy.

From personal experience - I have written a number of programs from scratch -- but all too often some hack would do the same thing by just copy/pasting something they got and changing a few things. The difference becomes only clear if both parties are forced to think and program for themselves.

When an algorithm becomes "common knowledge" then it becomes no longer necessary to code that from scratch.. And i believe this is what has to be determined.
What is common knowledge that is OK to copy and what is not.

[Don]

I think this is just an opinion, and of course you have the right to have your own opinion. But I don't share it. I see no logical need why a contest would imply it is between authors. I consider my engines like my children. And I would consider it very unfair when the organizers of a Chess tournament would say to my second born: "sorry, we don't want you, because your brother already plays in this championship, and we allow only one sibling per family". Or that they say to my eldest: "sorry, we already have registered your younger brother, because he is stronger". When I was in highschool, both my brother and me were allowed to participate in the same championship.

I did not state my opinion on whether this is how it should be, I'm just saying this it is how it really IS and that is not an opinion.

I have played in a lot of computer chess tournaments and in most of them there was some kind of prize that was awarded. Often a trophy but also I have received money. I did not give the money to my program, I kept it for myself! The money and/or trophy was awarded to ME, not my program. I think that is irrefutable evidence that this is how people think about these tournaments.

Now, where opinion comes into the picture is whether you or I believe this is healthy. Do we believe this is how it should be? That would be an opinion and we might agree or disagree on this point. But it has nothing to do with how it actually IS.

When the computer chess world championships are held, it's strictly one program per author. If this is really a contest between programs (with no ego's involved) then it would make sense to allow people to bring as many different versions as they wish. Do you think that is likely?

There is even debate over opening books, should the same opening book author be allowed to support many different programs? That has been discussed here. If this were really a contest between software/hardware entities this would not even be an issue, but I think this too proves that people look at these things as author contests.

I'm not saying you cannot choose to view them as contests strictly between programs, but if you do you should realize that they are not constructed to be fair from that standpoint. I cannot co-author 2 different programs for instance and that handicaps a given program unfairly.

You are confusing how you or I think it should be, with how it really is. It might even be the case that I agree with you on this. In an ideal world perhaps that would be a healthier way to view things and more productive in the long run. Unfortunately, the "cloners" are not taking this point of view either even though they sometimes pretend to. They view their work as either a personal triumph that they take credit for, or as a vendetta against someone else's way of life.

That doesn't mean I can't feel proud if my children accomplish something great. Part of the glory of children rubs off on their parents. If anything, it proves they are good breeding stock.