Tournament Practice: Similarity Limits (Basic Discussion)

[Kirill Kryukov]

One possible solution is to find some objctive method to measure engine-engine similarity. For example ponder hit statistics. Setting some border line value based on existing analysis like CCRL.

Problems with this method:

1. There will be always discussion about what level of ponder hit is acceptable to say that two engines are independent. From my observation I woud say somewhere around 68% or 69%. This should be determined on large number of games.

2. Theoretically two independent engines can have similar playing styles by chance. Though in all cases of high ponder hit known to me I tend to think it is not coincidental.

3. In some cases of high similarity (high ponder hit) it will be difficult to prove which one is the original. Perhaps the one with open source out of two. If both are closed source then it becomes difficult. Also Strelka case shows that open source is not necessarilty a proof of originality.

[Tony]

People actually showing up at tournements have a pretty common opinion.

That is not surprising if people with a different opinion are excluded from showing up.

Ah excellent !!!

The programmers that support clones to participate you mentioned are the cloners themselves .

Brilliant.

Maybe you know some thieves that think stealing should be allowed ?

Tony

[Uri Blass]

One possible solution is to find some objctive method to measure engine-engine similarity. For example ponder hit statistics. Setting some border line value based on existing analysis like CCRL.

Problems with this method:

1. There will be always discussion about what level of ponder hit is acceptable to say that two engines are independent. From my observation I woud say somewhere around 68% or 69%. This should be determined on large number of games.

2. Theoretically two independent engines can have similar playing styles by chance. Though in all cases of high ponder hit known to me I tend to think it is not coincidental.

3. In some cases of high similarity (high ponder hit) it will be difficult to prove which one is the original. Perhaps the one with open source out of two. If both are closed source then it becomes difficult. Also Strelka case shows that open source is not necessarilty a proof of originality.

I suspect that it may be possible to change the evaluation of fruit and get less than 68% when the new version is not more than 50 elo weaker than fruit.

Uri

[Rolf]

It seems that the programmers have a common opinion.

It seems that you have read statements of some programmers only, and ignored statements of other programmers. But that is normal. Nobody likes to read statements which disagree with his own opinion.

Which programmers stated that clones should participate at WCCC ?

Reread what I wrote. People actually showing up at tournements have a pretty common opinion.

They just seem to make a lot less noice than the ones who don't.

Tony

This is not directly connected but may I ask the following:

for me, just from the outside, I would find that it's unfair if for a tournament a special "winner-killer" is secretly applauded by the broad mass of the other programmers. I know that it was always fun for others to have I think it was Gandalf attending the event for a possible killer point against Fritz. Wouldnt you potentially see a little unfairness in such a an incident? Not a personal question to you but a general to all. Would be nice if someone could give me either link or explanation why such a Toga Cluster (?) was allowed for Paderborn?

[Rolf]

One possible solution is to find some objctive method to measure engine-engine similarity. For example ponder hit statistics. Setting some border line value based on existing analysis like CCRL.

Problems with this method:

1. There will be always discussion about what level of ponder hit is acceptable to say that two engines are independent. From my observation I woud say somewhere around 68% or 69%. This should be determined on large number of games.

2. Theoretically two independent engines can have similar playing styles by chance. Though in all cases of high ponder hit known to me I tend to think it is not coincidental.

3. In some cases of high similarity (high ponder hit) it will be difficult to prove which one is the original. Perhaps the one with open source out of two. If both are closed source then it becomes difficult. Also Strelka case shows that open source is not necessarilty a proof of originality.

That sounds to at least somehow addressing my original question. Thanks for your input so far.

[Mike S.]

Maybe you know some thieves that think stealing should be allowed ?

Programmers who use open source code in compliance with the license, and contribute to it's further developement, aren't thieves.

I have never claimed that illegal clones should participate anywhere. Strelka of course is something entirely different in terms of legality and ethics, than Toga. - But that is what I meant: Obviously it's much too complicated too distuingish between these types of (so called) clones. Appearantly, the computer chess gurus can only handle (1.) ban them all, or (2.) allow anything... But that is stupid and not justifyable. This needs some fine tuning.

[Rolf]

Maybe you know some thieves that think stealing should be allowed ?

Programmers who use open source code in compliance with the license, and contribute to it's further developement, aren't thieves.

I have never claimed that illegal clones should participate anywhere. Strelka of course is something entirely different in terms of legality and ethics, than Toga. - But that is what I meant: Obviously it's much too complicated too distuingish between these types of (so called) clones. Appearantly, the computer chess gurus can only handle (1.) ban them all, or (2.) allow anything... But that is stupid and not justifyable. This needs some fine tuning.

Not legally but ethically for sports events. Look, if I could get the genetical code of Maradonna, then of course I can add certain German adds but if I then would be allowed to play for the German team against Austria it would still appear to be odd for you. And forbidden? I cant understand why people like you could enjoy someone taking another ones's work and participating at a tournament when he on his own had no chance to compete... All after the motto that everything what isnt forbidden is allowed? I dont buy such illogic.

P.S. I know that this is dangerous to think but IMO it's also unethicsal to pretend that a machine program could play decent chess when it relies on books of GM chess. Because without these books the progs would be buried against preparing humans. Why do we pretend that a dumb machine is also preparing things when in truth it just steals from human made code. I told a computational scientist, the best were that a machine would play a tourney completely on its own, the games of the rounds added of course. She said, but it should be told against whom it's playing. I said, but this can be faked like a human could prepare a different style of a machine... Unless we dont let the machine play its own chess it's all about pretending, impostering. Is it not?

[BubbaTough]

it's also unethicsal to pretend that a machine program could play decent chess when it relies on books of GM chess.

errr...LearningLemming does not use books of GM chess, and at least in CCT10 did not have any bad positions out of "book", even playing against top engines with solid books such as Rykba, Junior, and Glaurung. It may have only score 0.5 / 3 against these engines, but that had nothing to do with the books .

Also, LearningLemming plays online on ICC against humans quite a bit, and does just fine in the opening against strong humans without a book of GM chess.

-Sam

[Uri Blass]

Maybe you know some thieves that think stealing should be allowed ?

Programmers who use open source code in compliance with the license, and contribute to it's further developement, aren't thieves.

I have never claimed that illegal clones should participate anywhere. Strelka of course is something entirely different in terms of legality and ethics, than Toga. - But that is what I meant: Obviously it's much too complicated too distuingish between these types of (so called) clones. Appearantly, the computer chess gurus can only handle (1.) ban them all, or (2.) allow anything... But that is stupid and not justifyable. This needs some fine tuning.

Not legally but ethically for sports events. Look, if I could get the genetical code of Maradonna, then of course I can add certain German adds but if I then would be allowed to play for the German team against Austria it would still appear to be odd for you. And forbidden? I cant understand why people like you could enjoy someone taking another ones's work and participating at a tournament when he on his own had no chance to compete... All after the motto that everything what isnt forbidden is allowed? I dont buy such illogic.

P.S. I know that this is dangerous to think but IMO it's also unethicsal to pretend that a machine program could play decent chess when it relies on books of GM chess. Because without these books the progs would be buried against preparing humans. Why do we pretend that a dumb machine is also preparing things when in truth it just steals from human made code. I told a computational scientist, the best were that a machine would play a tourney completely on its own, the games of the rounds added of course. She said, but it should be told against whom it's playing. I said, but this can be faked like a human could prepare a different style of a machine... Unless we dont let the machine play its own chess it's all about pretending, impostering. Is it not?

Rybka already beat GM Ehlvest with a book of only 3 moves
so book preperation is not important against humans.

Prepared human may win against some deterministic machine with no book by trying many ideas until he find something that works and repeat it but this is not interesting because it is enough to repeat machine-machine game in order to do it.

Uri

[Rolf]

it's also unethicsal to pretend that a machine program could play decent chess when it relies on books of GM chess.

errr...LearningLemming does not use books of GM chess, and at least in CCT10 did not have any bad positions out of "book", even playing against top engines with solid books such as Rykba, Junior, and Glaurung. It may have only score 0.5 / 3 against these engines, but that had nothing to do with the books .

Also, LearningLemming plays online on ICC against humans quite a bit, and does just fine in the opening against strong humans without a book of GM chess.

-Sam

Thanks for such a feedback. I belong to those who cant follow every new development. But this here and also from Uri strikes me. Only 3-move book? Interesting.