Murka 3.0 released

[velmarin]

*Especialy not in this case as the source is available. It is a bit like having a video of a crime but judging on a drunken witness

exactly, you said.

I did not want him to say,
but these Puritans, not programmers.
The code is there, just seem stupid.

[michiguel]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Bold, but true. Whatever positive has been produced so far, it was not demonstrated to be false. There was always a reason for the hit. Not necessarily the reason was "cloning".

Miguel

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

[michiguel]

Hi Ingo,

Your reaction was my first reaction. But I'm now intrigued.

Up to now I have had little interest in detecting clones. Simply because clones have zero interest for me. But this seems interesting.

If the positions are truly random with multiple "good" or reasonable moves then it is suspicious if there are a high percentage of common choices. Since the test is done with little time per position I really think it's a test of cloned evaluation functions. I cannot see how search plays too much of a role (I could be wrong).

My one point of skepticism is maybe the test is simply detecting similar styles and not just clones. So I could create an engine which likes to throw it's pieces towards the enemy's king. This (surely) increases the likelihood the engine will have a higher common hit rate with other aggressive engines. But I remain open minded.

Steve

The test has taught us that style is intrinsically related to a given engine. The test do measure style, and as such, it is very powerful to detect style similarities. Some people like to have engines that diverge, to have different options. Until today, this was based on opinions. Not anymore, now you can measure it.

Miguel

[Adam Hair]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

Hi Ingo,

That is why I used the word "seemingly". As far as anyone knows (= seemingly), the similarity test has not produced a false positive.

I do not think that this tool can prove anything either. But I am certain it is very good at showing where to look.

Adam

[Graham Banks]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

Hi Ingo,

That is why I used the word "seemingly". As far as anyone knows (= seemingly), the similarity test has not produced a false positive.

I do not think that this tool can prove anything either. But I am certain it is very good at showing where to look.

Adam

The English language can be unkind to those for whom it is not their native tongue.

[Adam Hair]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

Hi Ingo,

That is why I used the word "seemingly". As far as anyone knows (= seemingly), the similarity test has not produced a false positive.

I do not think that this tool can prove anything either. But I am certain it is very good at showing where to look.

Adam

The English language can be unkind to those for whom it is not their native tongue.

I hope that Ingo does not think that I was being rude. I only wanted to point out that I was not trying to be as bold as he thought I was.

[Adam Hair]

Positions are chosen such as to not have a single best move.

I thought they were hundreds of random positions.

No, they are well filtered neutral positions, which have no clear best move, and unrelated engines often disagree on. There are positions on which even perfect engines can choose different moves, as shown by TBs.

Who filtered them?

Isn't this Sim by Don? In this case Don filtered them, 8,000+ positions.

I wonder how much time he required to do that. And what purpose? Could have been spent on Komodo development.

BTW, I would not be surprised if any author filtered out 8000 positions and it turned out that his own engine is original.

If I remember correctly, the positions came from a set that Larry and Don originally used in a different context.

As far as the possibility that the positions were filtered so that Komodo would look original, the tool is built in such a way that the positions can be substituted with other positions. Don actually helped me figure out how to do it. Plus, I have compared various engines (including Komodo) with other sets of positions. Don did not filter the positions in such a way that made Komodo look original.

[Adam Hair]

By the way, Ed Schröder did an analysis of the original 8239 positions. The following table shows the number of different moves selected by the engines in Ed's study for the positions. 1191 positions had the same move selected by all the engines; 3049 positions had two possible moves according to the test engines; and etcetera:

Code: Select all

 1 move     1191   (14.46%)
 2 moves    3049   (37.01%)
 3 moves    2243   (27.23%)
 4 moves    1151   (13.97%)
 5 moves     486    (5.90%)
 6 moves     101    (1.23%)
 7 moves      14    (0.17%)
 8 moves       3    (0.03%)
 9 moves       0    (0.00%)
10 moves       0    (0.00%)
11 moves       0    (0.00%)
12 moves       0    (0.00%)

If you read the link, you will see that the original positions were compared to a different set of positions. Due to the characteristics of the second set, the similarity percentages were much higher across the board. That shows something that I have tried to make clear. The percentage of matched moves requires context to be interpreted. I believe that the 60% threshold originated with me. I chose that percentage based on the results I accumulated on my computer under my conditions in such a way to minimize the chance of a false positive*. It is not necessarily true that 60% has the same significance in other conditions, such as other computers or other positions. In other words, some care must be taken in interpreting the results from a single test.

*It has been put forward that I was too conservative in assigning 60% as the threshold for determining a possible derivative.

[jshriver]

I look forward to testing this engine. Congrats on the release!

-Josh

[IWB]

Though the similarity test seemingly has not produced any false positives,

Hi Adam,

That is a bold statement. "We do not know of false positives" would be better!

Anyhow, I don't like the test at all as it is used for a witch hunt. Just have a look at this thread. Even if the source is available anyone not knowing anything about programming can use it to throw stones. But that is something you already mentioned.

I personally do not think that this tool is good to prove anything. If there IS something like "perfect play" it might give a 100% similarity. Similarity is true but not saying that the two programs are identical. So, the better the programs get the more similar they might be ...

Bye
Ingo

Hi Ingo,

That is why I used the word "seemingly". As far as anyone knows (= seemingly), the similarity test has not produced a false positive.

I do not think that this tool can prove anything either. But I am certain it is very good at showing where to look.

Adam

The English language can be unkind to those for whom it is not their native tongue.

I hope that Ingo does not think that I was being rude. I only wanted to point out that I was not trying to be as bold as he thought I was.

No offense taken, however, I was aware that "seemingly" is weakening the content of the sentence but still think that it is too much excluding the possibility of a false positive.

Anyhow, my problem is more the tool itself then the sentence.

BYe
Ingo