2. Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.
I would like to see a clarification of the "or the source of such code".
The exisiting rule explicitly allows "inclusion of game-playing code written by others", and thus apparently does not consider such code by definition "non-original work". It is just an orthogonal property, code written by others can be an original work (or not) of these others, and this is apparently what matters.
The rule does state I must list the name of these others in the submission details. It does not seem to require that these others actually sign in person. Is there another rule that requires that?
In any case it offers an alternative to giving the names: it is enough if I just say how I obtained the code. This does not seem to exclude code from anonymous authors. Of course common law in that case would force me to show that I can legally use that code, but a license agreement that came with the code would satisfy that requirement.
I don't know if this is all intended, or just an unforseen side effect. But I would like to see it made explicit that legal use of open-source code is in your program is not a violation of rule #2 per se, but will be treated as any other code written by a co-author that has given permission to enter it in the ICGA.
This will require that there also must be a clarification as to what should happen when two entrants want to use the same code. (Which can also happen when former collaborators fall out with each other, and continue development on their own. So a rule for this is needed anyway.) I would propose that in such a case the code can only be used by one of the participants, and that in cases where the would-be participants cannot agree about which of them will enter, they can be obliged to play a qualifyer match to decide this.
Ok, to make it simple, we just use the similarity test. If a programmer objects to its results, he may protest by providing objective evidence to a panel of experts.
So as long as the code passes SIMTEST, it should be allowed.
As to what should be the cut off, that can be debated. I would say < 65% should be allowed.
That sounds like a very bad idea. It is only a matter of time before people will learn how to thwart the simtest, and then you basically give them permission to do whatever they want.
There must be time-resistent objective rules of what is allowed. The simtest could at best be a tool used to unmask cheaters that violate the objective rules.
hgm wrote:That sounds like a very bad idea. It is only a matter of time before people will learn how to thwart the simtest, and then you basically give them permission to do whatever they want.
There must be time-resistent objective rules of what is allowed. The simtest could at best be a tool used to unmask cheaters that violate the objective rules.
If you have a better test, then please share it.
Again I emphasize that idealism should be avoided. If you are going to wait for the "magical utility/rule" that is time-resistant and "un-thwartable", then you can basically wait forever and the WCC will never happen.
One must be able to compromise sometimes and just move on.
Yes, there will be people who will game the similarity test. They will be able to play for a while, but soon they would be caught, and allegations will start to fly in all directions. Then we can see what to do. They would probably have to send in objective evidence to prove their originality.
bob wrote:
It sounds like you advocate "all derivatives are OK except if they come from IPPOLIT?"
To date, ippolit has not been proven to be a rybka derivative, so what makes it special here? My issue is that I believe ippolit is not original, based on looking at the source code which appears to come from hex rays or something similar. And we have no author to question.
But is it really ok to have fruit derivatives, crafty derivatives, stockfish derivatives, any other open-source derivatives?
No of course not. If a program is derived from another program and it violate's its license, it should not be allowed. Thus, in the case of open-source derivative programs, if no license violation has occured, and the program passes the similarity test, it should be allowed to participate.
Note: I only mentioned IPPOLIT as special case because it is "public domain" but is unoriginal and illegal derivative of rybka (unproven, but highly suspicious). Again, I must emphasize that we can debate about IPPOLIT all day and never come to any scientific conclusion, so lets not get idealistic. It is commonly accepted in the computer chess community that IPPOLIT is an illegal derivative, so lets not waste time about it and just move on. As long as it passes the similarity test, an IPPOLIT derivate should be fine. If not, it should not be allowed.
What about question #2. Ok to have 30 stockfish derivatives? Again, that is not something I want to do. I could run that on my cluster, for example.
2. Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.
I would like to see a clarification of the "or the source of such code".
The exisiting rule explicitly allows "inclusion of game-playing code written by others", and thus apparently does not consider such code by definition "non-original work". It is just an orthogonal property, code written by others can be an original work (or not) of these others, and this is apparently what matters.
The rule does state I must list the name of these others in the submission details. It does not seem to require that these others actually sign in person. Is there another rule that requires that?
In any case it offers an alternative to giving the names: it is enough if I just say how I obtained the code. This does not seem to exclude code from anonymous authors. Of course common law in that case would force me to show that I can legally use that code, but a license agreement that came with the code would satisfy that requirement.
I don't know if this is all intended, or just an unforseen side effect. But I would like to see it made explicit that legal use of open-source code is in your program is not a violation of rule #2 per se, but will be treated as any other code written by a co-author that has given permission to enter it in the ICGA.
This will require that there also must be a clarification as to what should happen when two entrants want to use the same code. (Which can also happen when former collaborators fall out with each other, and continue development on their own. So a rule for this is needed anyway.) I would propose that in such a case the code can only be used by one of the participants, and that in cases where the would-be participants cannot agree about which of them will enter, they can be obliged to play a qualifyer match to decide this.
The intent was to address several cases. Toga, for example, a derivative of Fruit. Or grid chess, a derivative of Crafty. Etc. ICGA rules would allow that, so long as both the derivative and its parent did not both try to enter. The reason for asking for the source of the original code was to limit a family to one participant, by tracking provenance of the various versions.
hgm wrote:That sounds like a very bad idea. It is only a matter of time before people will learn how to thwart the simtest, and then you basically give them permission to do whatever they want.
There must be time-resistent objective rules of what is allowed. The simtest could at best be a tool used to unmask cheaters that violate the objective rules.
If you have a better test, then please share it.
Again I emphasize that idealism should be avoided. If you are going to wait for the "magical utility/rule" that is time-resistant and "un-thwartable", then you can basically wait forever and the WCC will never happen.
One must be able to compromise sometimes and just move on.
Yes, there will be people who will game the similarity test. They will be able to play for a while, but soon they would be caught, and allegations will start to fly in all directions. Then we can see what to do. They would probably have to send in objective evidence to prove their originality.
I have no qualms about using the similarity tester. My only disagreement is Ed likes to use the term "proof". Passing simtest proves nothing. Failing simtest proves nothing. Either suggest something, to be sure, but proof can only come from an actual source comparison. Given that, it is useful...
hgm wrote:That sounds like a very bad idea. It is only a matter of time before people will learn how to thwart the simtest, and then you basically give them permission to do whatever they want.
There must be time-resistent objective rules of what is allowed. The simtest could at best be a tool used to unmask cheaters that violate the objective rules.
I must add that any attempt so far to thwart the similarity test (such as Uri's Twinfish) has come at a great cost in elo. It will be very difficult for a participant to bypass the test in the most crudest of fashions and win the championship at the same time.
There will always be cheaters and you have to just deal with it. Such is the nature of computer programming. In an ideal world, all source code would be open and everything would be transparent, but it is not.
I urge everyone to get practical now, shun your idealism and arrange for a good WCC tournament before the end of this year. I presume that should not be a difficult goal.
hgm wrote:That sounds like a very bad idea. It is only a matter of time before people will learn how to thwart the simtest, and then you basically give them permission to do whatever they want.
There must be time-resistent objective rules of what is allowed. The simtest could at best be a tool used to unmask cheaters that violate the objective rules.
If you have a better test, then please share it.
Again I emphasize that idealism should be avoided. If you are going to wait for the "magical utility/rule" that is time-resistant and "un-thwartable", then you can basically wait forever and the WCC will never happen.
One must be able to compromise sometimes and just move on.
Yes, there will be people who will game the similarity test. They will be able to play for a while, but soon they would be caught, and allegations will start to fly in all directions. Then we can see what to do. They would probably have to send in objective evidence to prove their originality.
I have no qualms about using the similarity tester. My only disagreement is Ed likes to use the term "proof". Passing simtest proves nothing. Failing simtest proves nothing. Either suggest something, to be sure, but proof can only come from an actual source comparison. Given that, it is useful...
Well, then Ed is an idealist, and listening to his demands are not going to get us anywhere. I think we must be practical.
bob wrote:
It sounds like you advocate "all derivatives are OK except if they come from IPPOLIT?"
To date, ippolit has not been proven to be a rybka derivative, so what makes it special here? My issue is that I believe ippolit is not original, based on looking at the source code which appears to come from hex rays or something similar. And we have no author to question.
But is it really ok to have fruit derivatives, crafty derivatives, stockfish derivatives, any other open-source derivatives?
No of course not. If a program is derived from another program and it violate's its license, it should not be allowed. Thus, in the case of open-source derivative programs, if no license violation has occured, and the program passes the similarity test, it should be allowed to participate.
Note: I only mentioned IPPOLIT as special case because it is "public domain" but is unoriginal and illegal derivative of rybka (unproven, but highly suspicious). Again, I must emphasize that we can debate about IPPOLIT all day and never come to any scientific conclusion, so lets not get idealistic. It is commonly accepted in the computer chess community that IPPOLIT is an illegal derivative, so lets not waste time about it and just move on. As long as it passes the similarity test, an IPPOLIT derivate should be fine. If not, it should not be allowed.
What about question #2. Ok to have 30 stockfish derivatives? Again, that is not something I want to do. I could run that on my cluster, for example.
30 stockfish derivatives? Again, if they pass the simtest, then they are through. I highly doubt though that they would.
bob wrote:First, here is the current ICGA Rule 2, which is regarding originality:
2. Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.
It is my understanding that Levy rewrote them for the Yokohama tournament.
When his changes were discussed, your feedback was that there was no need for change. Good to see your mind has changed. The new rules are not much better IMHO, but it is good that it is recognised that something needed to be done about them and a step was taken.
Last edited by mvk on Sat Feb 22, 2014 12:49 am, edited 7 times in total.
