Probably i will try it, it sounds interesting, but even if i do, i will consider this more like a style similarity test instead of cloning one.Don wrote:I believe the only reliable measure is the actual move played because everything else can be faked. But you cannot fake the move.xcomponent wrote:Hi guys!
Wouldn't be more correct if the test includes the similarities in the pv as well.
A single move could misleads since the reason for picking it could be even a move ordering bug. I know that huge amount of positions could suppress these kind of possibilities, but they are still remains. I think that the test make sense, but in a perfect world, could be more correct prove if it is combined with the pv similarities and the evaluation value.
But even if it were not faked, I'm not sure the PV is very reliable. In my own program the PV changes frequently, even if the first move does not. My guess is that each successive move is increasingly unreliable as a measure of similarity. And I admit that I'm guessing here, but my sense of this is that even if it was an improvement it would be only a very minor one. And as I mentioned, it can still be faked.
But there is no need to guess, try the experiment yourself, it was very easy to construct - and see if you can produce a better measure using many moves of the PV.
Clone detection test
Moderator: Ras
-
- Posts: 454
- Joined: Sat Apr 04, 2009 6:44 pm
- Location: Bulgaria
Re: Clone detection test
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Clone detection test
Don't get hung up on my Subject line, "Clone detection test", that was just a teaser to suck everyone in.
Larry did a similar thing recently on some forum, he made a post that implied Rybka was in trouble due to Komodo. While I would like to think that might happen someday, the title was merely a teaser, Rybka is still much stronger than Komodo.
Larry did a similar thing recently on some forum, he made a post that implied Rybka was in trouble due to Komodo. While I would like to think that might happen someday, the title was merely a teaser, Rybka is still much stronger than Komodo.
-
- Posts: 454
- Joined: Sat Apr 04, 2009 6:44 pm
- Location: Bulgaria
Re: Clone detection test
I had my suspicion on that 
Regarding the previous posts, as a long time assembly programmer, i can say that the distance from reversing to assembler could be like from earth to the moon especially after a couple of compiler optimizations. It takes not only assembly knowledge, but a lot of free time and efforts which I in particular don't have.

Regarding the previous posts, as a long time assembly programmer, i can say that the distance from reversing to assembler could be like from earth to the moon especially after a couple of compiler optimizations. It takes not only assembly knowledge, but a lot of free time and efforts which I in particular don't have.
-
- Posts: 1822
- Joined: Thu Mar 09, 2006 11:54 pm
- Location: The Netherlands
Re: Clone detection test
And how many 'good C programmers' is trying to find clones huh?Don wrote:Disassembled code looks different on different compilers and requires an expert. It's easy for you and I, but not everyone. A lot of good C programmers do not know assembler.diep wrote:I didn't follow the thread, but what's all this idiocy man?Don wrote:I think you are being overly concerned. I have seen over the years many cheaters exposed (or let's just say misunderstandings cleared up) simply because the author of a program noticed that a program in a tournament was playing like his.Dann Corbit wrote: I am beginning to think more and more that it is perhaps not a great idea to accuse someone of something very bad because his engine plays similarly to another engine. But it is also possible that some magic formula will occur that is foolproof. In any case, the idea makes me very nervous.
For instance several years ago I am playing at the Dutch Computer Chess Championship and get an email from John Stanback - who is watching the games from the states and asks me to check into a problem, he notices that one of the program is playing just like Zarkov.
In another tournament Richard Lang is sitting across from a clone of his own program (in this case it's a perfect clone) and he notices that a program is just too similar to his even though it's disguised in a different housing (it's one of those hardware chess computers.)
In yet another tournament Bob Hyatt notices remotely that one of the contestants is playing exactly the same moves as Crafty.
As a result of these observations, the problems in each case were investigated and were brought to some kind of resolution. Just noticing the similarity itself was not considered the proof but it was good enough to start asking questions.
So just relax - if we build this tool it will be with the understanding that it's imperfect - it's just a crude measurement. Like almost any tool, it's not a bad thing but someone could grab a screwdriver and use as a weapon to hurt someone with, it doesn't mean we should not have screwdrivers.
So I believe this is a powerful test but like you I agree that it should not be used as a weapon to hit someone over the head with.
I could build a simple UCI test harness to run my test and produce a result file - anyone interesting in doing some kind of blind test?
Don
Just install IDA pro and look in the assembler code of an engine and you know instantly whether it's a clone.
And nothing else can prove anything unless the guy who clones is a major idiot; note majority is major idiots to copy things 100%. With source code available now it's easy to make some modifications modifying behaviour.
So just take a look in the assembler code of the engine and you know it all. Simple as that.
Thanks,
Vincent
The similarity testers would be a tool and nothing more. It would be used in conjunction with other things, such as the disassembler.
Don
Let's face it, there's a lot of clones out there that copy the then best engine they have access to, and nearly 0 of those programmers ever posted a word about it, they're all hiding themselves under 100 meters of sand.
However in that shelter 100 meters deep down they complain major league about what happens and want someone else, usually me, to post the bad news.
Naum huh?
Did you take a look in its assembler already?
Every algorithm and thingie the same as rybka. Why then have an executable that at 32 bits is same speed as at 64 bits (tested by Andre van Ark) and with sureness be 50-100 elo weaker or something than rybka3?
Contractual appointments huh?
Every programmer knows it for years already and 0 have posted it so far.
So who is going to use your 'clone detection test' ?
It's again wasted time.
From the guys who post here in this thread, about half of them knows already more clone names, that officialy have not been 'uncovered'. So why do they cooperate with a clone test that actually proves nothing.
Even IF you have similarity, there has already been evidence that it is possible to train a chessprogram to *behave* like another program quite ok.
"oh and which engine is that?"
Well it's not Diep and it kicked hell out of a few other engines in a world champs one day, despite running single cpu.
There happens so much there already that has been accomplished, that it's already impossible to follow.
Is trying to copy behaviour illegal or is that also falling under 'clone law'?
As that's what todays sophisticated self learners are doing.
As for fresh engines, todays optimization techniques and testing techniques needed to produce an engine that's 2700+, they are so difficult mathematical, that no unknown person will ever manage to do so.
Anyone who just shows up and has 2700+ engine, it's a clone, stolen concept, or someone who produced it under a fake name, or someone who bought a license.
Speaking of licenses - i assume that is legal.
How legal is it that several authors buy a license, one is called rybka, another one has another name and goes under other names.
Isn't it in fact also hard fact that a lot of those engines are in face originating from the same codebase and therefore legally are the same engine?
So only 1 of them may join the world champs or?
So many seems to have bought, or illegally manufactured, their own rybka, long before the ippolit source code had been put online, that it's just sick.
Thinker huh?
As if this guy knows anything about computerchess "suddenly".
In assembler the proof already has been delivered that he tried to obfuscate it a little, but it definitely is another case of thievery. Or did it legally get licensed?
You know, all programmers , especially the C programmers, also realize this, that he can't get that strong "out of the blue sky". In assembler it's visible. If you of course obfuscate your engine and don't show the mainline, just let it play a move, of course every idiot on the street realizes why that is. It has been weakened by using smaller hashtable sizes, which youc an't set to some decent amount, say a couple of hundreds of MB, why is THAT?
Because that makes an engine a lot weaker, and entire computerchess community already knows this for a year or 15+.
Well - except for Christophe Theron then that is (i remember start of 21th century a beta version he had shipped to Jeroen to use at a tournament, which used a very tiny hashtable of a few hundreds of kilobyte also).
Deliberate, according to contract, weakened engine again?
As it may not beat rybka3 again?
You know, everyone just shuts up always. They all know what's going on, but no one talks.
So why design a test if that's not solving the core of the problem?
Vincent
-
- Posts: 588
- Joined: Sun Nov 23, 2008 11:16 pm
- Location: Berlin, Germany
Re: Clone detection test
Would anyone mind if this thread got relocated to the forum dedicated to Engine Origins, since that's ultimately what it's about? Seems thematically appropriate. Please PM me if you have objections.
Thanks
Jeremy
Thanks
Jeremy
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Clone detection test
I have an objection because this particular thread is of a technical nature more than political. I view that other forum as non-technical and political and it probably would not attract the more serious engineers as much as this one would.sockmonkey wrote:Would anyone mind if this thread got relocated to the forum dedicated to Engine Origins, since that's ultimately what it's about? Seems thematically appropriate. Please PM me if you have objections.
Thanks
Jeremy
Besides, the subject of this thread was specifically designed to be provocative, but it's really about designing instrumentation to measure the playing style of chess programs.
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: Clone detection test
I agree with Don.Don wrote:I have an objection because this particular thread is of a technical nature more than political. I view that other forum as non-technical and political and it probably would not attract the more serious engineers as much as this one would.sockmonkey wrote:Would anyone mind if this thread got relocated to the forum dedicated to Engine Origins, since that's ultimately what it's about? Seems thematically appropriate. Please PM me if you have objections.
Thanks
Jeremy
Besides, the subject of this thread was specifically designed to be provocative, but it's really about designing instrumentation to measure the playing style of chess programs.
Miguel
-
- Posts: 388
- Joined: Sun Dec 21, 2008 6:57 pm
- Location: Washington, DC
Re: Clone detection test
Ok, so I feel compelled to ask a couple of stupid questions...
How does this 'cloning' occur? Do the cloners have access to the source code, or do they decompile the executable and then work with the horrible mess that comes out? And what the hell's the point?
How does this 'cloning' occur? Do the cloners have access to the source code, or do they decompile the executable and then work with the horrible mess that comes out? And what the hell's the point?
Re: Clone detection test
Thinker did not appear "out of the blue sky", but was quite a strong engine before any rybka was there to be copied.diep wrote: Thinker huh?
As if this guy knows anything about computerchess "suddenly".
In assembler the proof already has been delivered that he tried to obfuscate it a little, but it definitely is another case of thievery. Or did it legally get licensed?
You know, all programmers , especially the C programmers, also realize this, that he can't get that strong "out of the blue sky". In assembler it's visible. If you of course obfuscate your engine and don't show the mainline, just let it play a move, of course every idiot on the street realizes why that is. It has been weakened by using smaller hashtable sizes, which youc an't set to some decent amount, say a couple of hundreds of MB, why is THAT?
Because that makes an engine a lot weaker, and entire computerchess community already knows this for a year or 15+.
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Clone detection test
It's well known that many programs have been disassembled. In many cases this is not disguised, in others it is denied.Greg Strong wrote:Ok, so I feel compelled to ask a couple of stupid questions...
How does this 'cloning' occur? Do the cloners have access to the source code, or do they decompile the executable and then work with the horrible mess that comes out? And what the hell's the point?
The point of doing this is to dig for ideas that you cannot come up with on your own, or would require too much effort to come up with on your own.