Firebird 1.0 and 1.01: 180 games.

[lkaufman]

I don't call them clones, I call them derivatives, which I don't think is very controversial. I don't even take a position on whether they are illegal or should be banned; that comes down to a judgement call on how much a program has to be modified before it becomes legit, and perhaps on whether the code was obtained by decompiling or by improper means, again an issue I have no comment on. I'm more interested in why Robbo outperforms Rybka on head to head tests than on its origins, because that may be of direct help in improving Komodo. If the two were not closely related this would not be so interesting. I'm also interested in which one is truly stronger for analysis purposes, as that is important for a lot of my work other than on Komodo. At the very least, I think I've learned that it pays to get time control algorithms right, and that it pays to be pretty careful about slowing a program down with "smarts". Finally, it doesn't hurt for me to post here, because I post while waiting for Komodo test results.

[kranium]

I don't call them clones, I call them derivatives, which I don't think is very controversial.

?
yes, it's controversial...
there has not been a scrap of evidence to support your claim.

PS your answer does not address the 'conflict of interest' issue

[lkaufman]

I've posted enough evidence of the eval similarities already, I have no real interest in further proving what's obvious. In general I'll confine my posts to issues relating to why Robbo outperforms Rybka, I have no conflict of interest in trying to learn the truth about this. Of course if someone asks me a question politely, I'll try to answer it if I can. Finally, in a way I don't mind if Robbo et al prove to be genuinely the strongest engines, as since they are using pretty much my eval numbers (if not many of the eval terms), I consider myself a co-author of these programs (unintentionally of course) and so take some pride in their strength.

[kranium]

I've posted enough evidence of the eval similarities already

i must have missed your evidence...
or do you mean the back rank of the king PSQ table which is 'similar' to the rybka3 table you worked on?..yes that i saw, but everyone seemed to agree, it's incredibly weak.

[lkaufman]

By itself there is maybe a 1% chance someone would have come up with the same numbers by chance, but I posted some other items, and i could post plenty more if I had enough incentive to do so. It's not my battle though, I've already been paid for my work.

[frcha]

By itself there is maybe a 1% chance someone would have come up with the same numbers by chance, but I posted some other items, and i could post plenty more if I had enough incentive to do so. It's not my battle though, I've already been paid for my work.

From what you see so far with Robbo, would you say that the work done on Robbo is significant enough to be an innovation ?

That is you would agree - it could not have been trivial but actually quite difficult to rev. engineer Rybka and create a stronger engine?

And lastly, do you have a rough idea of how much of Rybka is in Robbo as in 40% , 60% etc.

Note - I am not disbelieving anything you say, and I do think you have the right to speak freely and refer to them as derivatives.

[lkaufman]

If Robbo was reverse engineered (rather than based on improperly obtained code) and then brought to its present level that was a major project. The search seems to have been modified somewhat, in every case to make Robbo faster at some cost in search quality. I believe this was done to get favorable results when playing Rybka; it may not be helpful against unrelated engines. Regarding the eval, much was left out, and in general the remaining terms had equations replacing tables, but the equations were designed to come as close as possible to duplicating the tables, so that the effect is a similar eval except when the missing terms come into play. This also sped up the program at some cost. I think the biggest gain may have come from the time control work; here it was easy to improve on Rybka. So I'm not convinced that Robbo is actually a stronger engine if you ignore the time management aspect, although it is better tuned for fast play. Certainly a lot of work went into Robbo, but modifying an existing program is a minor task compared to writing a whole new one as we are doing with Komodo. As for the percentage of similarity, it comes down to interpretation. The eval code will look quite different as it uses formulas, but the output is similar. The main point is that the search is basically Rybka's search, which is quite unique, with only parameters and other details changed. It makes no sense to quote a percentage, because although many details are different, the basic search clearly comes from Rybka. Whether this is "illegal" I can't say; you are allowed to use ideas but not code from decompiling other engines, and I don't know how similar the code has to be to be considered illegal.

[beram]

If Robbo was reverse engineered (rather than based on improperly obtained code) and then brought to its present level that was a major project. The search seems to have been modified somewhat, in every case to make Robbo faster at some cost in search quality. I believe this was done to get favorable results when playing Rybka; it may not be helpful against unrelated engines.

You don't read enough relevant testings:
for instance http://www.talkchess.com/forum/viewtopic.php?t=32514

Total Scoring
GGTJan10_4221_10+12 Amar+Ware+Sokolsky+Swiss+From+Sturm
Code:

1 2 3 4 5 6 7
1 Fireb 1B ** 5,5 6,5 9,5 2,5 9,5 2,5 7,5 4,5 9,5 2,5 9,0 3,0 50,5 / 72
2 Dp Ryb 3 6,5 5,5 ** 6,5 5,5 7,0 5,0 10,0 2,0 9,0 3,0 9,5 2,5 48,5 / 72
3 Stofi 1.6 2,5 9,5 5,5 6,5 ** 7,0,5,0 6,5 5,5 5,5 6,5 7,0 5,0 34,0 / 72
4 Zap Mx II 2,5 9,5 5,0 7,0 5,0 7,0 ** 7,0 5,0 6,5 5,5 6,5 5,5 32,5 / 72
5 Naum 4 4,5 7,5 2,0 10 5,5 6,5 5,0 7,0 ** 5,5 6,6 8.0 4,0 30,5 / 72
6 Dp Shr 12 2,5 9,5 3,0 9,0 6,5 5,5 5,5 6,5 6,5 5,5 ** 6,0 6,0 30,0 / 72
7 Dp Fri 11 3,0 9,0 2,5 9,5 5,0 7,0 5,5 6,5 4,0 8,0 6,0 6,0 ** 26,0 / 72

Regarding the eval, much was left out, and in general the remaining terms had equations replacing tables, but the equations were designed to come as close as possible to duplicating the tables, so that the effect is a similar eval except when the missing terms come into play. This also sped up the program at some cost. I think the biggest gain may have come from the time control work; here it was easy to improve on Rybka. So I'm not convinced that Robbo is actually a stronger engine if you ignore the time management aspect, although it is better tuned for fast play.


Certainly a lot of work went into Robbo, but modifying an existing program is a minor task compared to writing a whole new one as we are doing with Komodo.
Than why does it take Vas so long ???

As for the percentage of similarity, it comes down to interpretation. The eval code will look quite different as it uses formulas, but the output is similar. The main point is that the search is basically Rybka's search, which is quite unique, with only parameters and other details changed. It makes no sense to quote a percentage, because although many details are different, the basic search clearly comes from Rybka. Whether this is "illegal" I can't say; you are allowed to use ideas but not code from decompiling other engines, and I don't know how similar the code has to be to be considered illegal.

[lkaufman]

So you base the claim that Firebird is stronger than Rybka on a lead of 2 points after 72 games (at a fast tc)? The very fact that these are so close whereas head to head testing shows clear wins for the Robbo family supports my claim that Robbo's superiority largely vanishes against unrelated engines, and any that remains is easily attributed just to better TC. Let me know if you can point to any test where Firebird or any of the Robbo family has a statistically significant lead against a field of unrelated engines, preferably not at a blitz TC.

[yanquis1972]

larry, i'm quite sure paul watson/NATIONAL 12 did a series of games which showed a massive superiority of RLg3 over naum and shredder, and possibly others. this was albeit at 4+2.

i think - i'm quite sure - that while RL was tuned against rybka at fast TC, well, so was/is rybka, if we are to take vas's word. yet this does not seem to cripple it against unrelated engines.