What the computer chess community needs to decide

[Gerd Isenberg]

Otherwise, the history of open source in computer chess and its influence is quite old. Richard Lang had reported he got a push after studying Spracklens Sargon assembly sources from 1978, the Austrian Merlin team started with Chess 0.5 (Pascal) by Atkin/Frey published 1978 in BYTE. I think many started with Turbo Chess (Torbo Pascal) by Kaare Danielsen, and later GNU-Chess, i.e. Uli Türke's Comet.

This may be a very douche bag-ish question, but why is Comet not considered a GPL violator? It's clearly derived from the GPL program GNUChess, and is distributed without source.

Uli made no secret out of starting with GNU Chess, and probably the changes he made were huge. As far as I know, nobody questioned Comet derived from GNU Chess during those innocent times in the mid 90s.

[lkaufman]

I am not trash-talking you.

Yes, you are. Your incessant suggestions about Houdini are effective trash-talk.

Since I can't disassemble code and Don has no interest in doing so, the only way to get answers to this question is to ask others who have done so or to ask you directly.

This has been a consistent behaviour of yours (now and previously): pushing others to disassemble Houdini to know how or why it's stronger.
Is this behaviour in accordance with the ethical standards of the Komodo team?"

Since I am convinced that Houdini started life as a near-clone of a program made from disassembled Rybka code, I see nothing wrong with asking what other people have learned from disassembling Houdini code.

So I ask you, was the large speedup from early Houdini to 1.5 primarily due to an original idea or ideas, or was it due to applying known ideas from other programs that were not in early Houdini versions?

There are some new and improved evaluation terms in Houdini 1.5, and a more effective search through various techniques, nearly all of which are original. Your describing this as a "large speedup" is very inaccurate."

Thanks for your answer. If this is confirmed by the disassemblers then you will have my respect for a great achievement in computer chess. I must admit that I now use Houdini myself for analysis, as I regard it as the best available Rybka derivative. Since the evaluation in Rybka 4 changed quite a bit from my eval in Rybka 3, I actually find that the evals in all the Ippo family programs and Houdini are closer to my own than are the evals in Rybka 4, and so I prefer them!

As to the claim that Houdini is basically Ivanhoe, I don't know this myself, I just have no reason to disbelieve what Chris wrote. I do know that the evals are very similar to my evals in R3 but scaled down a bit.

Houdini is NOT basically Ivanhoe (and reading Chris C it's obvious that he knows very little about Ivanhoe or Houdini).
And I'm ready to discuss with you all 40 games of the recent TCEC Elite Match with respect to your repeated, but so far unsubstantiated claim that the Houdini evals are similar to R3.

I'll be glad to discuss eval comparisons in detail. But first can you comment on the scaling down of the eval in Houdini? I found a version of Houdini that someone made that reverses this scaling down (I don't know how they made it) which makes it more comparable to Ippo family. Can you tell us why you downscale the final eval for output?

Robert

Larry

[Houdini]

Since I am convinced that Houdini started life as a near-clone of a program made from disassembled Rybka code, I see nothing wrong with asking what other people have learned from disassembling Houdini code.

So your personal conviction that somebody else has disassembled Rykba 3 gives you the right to disassemble Houdini and take whatever you can find?
You're the co-author of engine #5 on the IPON ranking list and a high-profile member of the computer chess community, and you're actively requesting information obtained by disassembling engine #1. That's simply amazing... after this, I'll certainly know what to answer when someone accuses me of unethical behaviour .

Thanks for your answer. If this is confirmed by the disassemblers then you will have my respect for a great achievement in computer chess. I must admit that I now use Houdini myself for analysis, as I regard it as the best available Rybka derivative.

Surely you didn't really believe that the 50+ Elo progress in Houdini 1.5 is just by some smart compiling or godly tuning, as some seem to suggest?

Since the evaluation in Rybka 4 changed quite a bit from my eval in Rybka 3, I actually find that the evals in all the Ippo family programs and Houdini are closer to my own than are the evals in Rybka 4, and so I prefer them!

Thank you for confirming that the evals from Houdini 1.5 and Rybka 4 are very different.

But first can you comment on the scaling down of the eval in Houdini? I found a version of Houdini that someone made that reverses this scaling down (I don't know how they made it) which makes it more comparable to Ippo family. Can you tell us why you downscale the final eval for output?

Houdini's evaluations are an attempt to produce a score that is consistent across the stages of the game and to make a +1.0 eval correspond to approx. 90% chance of winning the game.
The hacked Houdini versions that are now circulating may have some of this balancing code disabled, and may no longer take into account certain elements relating to endgame tablebases, contempt and the stage of the game.

Robert

[lkaufman]

Since I am convinced that Houdini started life as a near-clone of a program made from disassembled Rybka code, I see nothing wrong with asking what other people have learned from disassembling Houdini code.

So your personal conviction that somebody else has disassembled Rykba 3 gives you the right to disassemble Houdini and take whatever you can find?
You're the co-author of engine #5 on the IPON ranking list and a high-profile member of the computer chess community, and you're actively requesting information obtained by disassembling engine #1. That's simply amazing... after this, I'll certainly know what to answer when someone accuses me of unethical behaviour .

Even if I didn't believe Houdini started as disassembled code, it is accepted that ideas from other programs may be used, but not code or numerical tables. If someone has learned of some new ideas by disassembly and is willing to share these ideas with the community, we all benefit together. We just don't do disassembly ourselves.

Thanks for your answer. If this is confirmed by the disassemblers then you will have my respect for a great achievement in computer chess. I must admit that I now use Houdini myself for analysis, as I regard it as the best available Rybka derivative.

Surely you didn't really believe that the 50+ Elo progress in Houdini 1.5 is just by some smart compiling or godly tuning, as some seem to suggest?

No I don't believe that. I believe that Houdini 1.5 contains at least one substantial improvement, maybe more. I note though that Ivanhoe seems to have experienced a similar large Elo jump between version 47 and 49 in roughly the same time period, and I wonder if this is coincidence or not. Another possibility is that you found that some of Stockfish ideas also work well in Houdini; this is of course completely legitimate if that is the case. You do acknowledge using SF ideas.

Since the evaluation in Rybka 4 changed quite a bit from my eval in Rybka 3, I actually find that the evals in all the Ippo family programs and Houdini are closer to my own than are the evals in Rybka 4, and so I prefer them!

Thank you for confirming that the evals from Houdini 1.5 and Rybka 4 are very different.

Yes, I have no reason to believe that you took anything from Rybka 4.

But first can you comment on the scaling down of the eval in Houdini? I found a version of Houdini that someone made that reverses this scaling down (I don't know how they made it) which makes it more comparable to Ippo family. Can you tell us why you downscale the final eval for output?

Houdini's evaluations are an attempt to produce a score that is consistent across the stages of the game and to make a +1.0 eval correspond to approx. 90% chance of winning the game.

I see. That's a very good answer. Since you are redefining 1.0 in this manner, it would have been desirable to state so before this, but better late than never! I rather like this definition myself. It does make comparison with Ivanhoe evals more difficult though.

The hacked Houdini versions that are now circulating may have some of this balancing code disabled, and may no longer take into account certain elements relating to endgame tablebases, contempt and the stage of the game.

Robert

Larry

[bob]

Today only a complete asshole would start coding a new engine from scratch, i.e., ignoring all the (best) available ideias and resources.

The two are not the same. You can start from scratch using all the available ideas together with yours. And you can start from a complete code and put in your ideas.

Nope. It's exactly the same thing as long as you agree the existing code you are basing is the best. Furthermore I strongly believe any experienced software developer agrees with this.

Here is one that does not. Copying source is not "software development" by the way, it is "software plagiarism". HUGE difference...

I respect your opinion but I don't see it that way and don't consider copying. Furthermore, modern sw development is heavily based on code-reuse, either yours or, in most cases, from other sources.

regards

Code reuse is good - as long as it is allowed based on existing copyright and license situation.

For most SW companies "code reuse" means to reuse existing *own* code mainly.

Sven

Code reuse is good - as long as it is allowed based on existing copyright and license situation.

For most SW companies "code reuse" means to reuse existing *own* code mainly.

Code reuse is not just good is strongly encouraged. Only short-minded SW companies would ignore all the existing **free** libraries and sources available at a mouse click.

Why do you keep talking about "free" or "public domain" libraries and sources? That is not the subject we are talking about here. Fruit can't be copied unless the copier follows the GPL requirements... A commercial program can't be disassembled, converted to C, then modified and used/distributed. So somehow you have veered away from the discussion at hand and jumped into a sub-set of software development without considering the larger picture where code-reuse only applies within a single company or product of theirs, not across companies, and certainly not across all instantiations of a particular software idea.

I'm afraid it is you who is not considering the big picture, instead you are focused only on your chess engine development just looking through your own glasses and somehow obsessed by the law. I ought to say, IMO, this is a strange position coming from a man of science but I accept and try to understand that.

Regards
Paulo

So a "man of science" should be ok with breaking into a competitor's lab and stealing his science? Or a "man of science" should be able to copy significant parts of someone else's work and publish it as their own, ignoring copyright law, ethics, etc?

I do not get the argument. Yes, it would be nice if everyone worked for the "common good". And some of us share every new idea we discover and do exactly that. And some don't. And some try to short-cut the time/effort part of the equation and simply copy and change and then say "this is my original work."

[Don]

Since I am convinced that Houdini started life as a near-clone of a program made from disassembled Rybka code, I see nothing wrong with asking what other people have learned from disassembling Houdini code.

So your personal conviction that somebody else has disassembled Rykba 3 gives you the right to disassemble Houdini and take whatever you can find?

I'll answer this question for myself. Many months (perhaps a couple of years ago) I was interviewed by Frank Quisinsky and made a statement about being willing to look at any sources I could find in order to get ideas for Doch (now Komodo.) I don't have any problem with IDEAS, we are all using almost exactly the same set of ideas anyway with only minor variations. So why isn't everyone writing incredibly strong programs?

Because it has a lot more to do with engineering than it does just throwing a bunch of ideas together. For instance we all use the same dictionary and know the same words, but not everyone is a great writer.

You pulled a Rosie Ruiz by jumping to the front of the line and starting from there. You started from a complete program that was already finely engineered to play incredibly strong chess - and it's not the least bit clear that you could have done that on your own.

Improving an already existing program is an order of magnitude easier than crafting one from scratch. I can watch the top tennis players and tell you when they make stupid mistakes - in affect there is something I could contribute to their game, but that does not mean I can duplicate their results for myself. I would not stand a chance because they are great tennis players and I am not.

You're the co-author of engine #5 on the IPON ranking list and a high-profile member of the computer chess community, and you're actively requesting information obtained by disassembling engine #1. That's simply amazing... after this, I'll certainly know what to answer when someone accuses me of unethical behaviour .

There is little to be gained by disassembling your program just in the search for idea to improve our programs. I suspect that if we were actively seeking to find new ideas we would not find anything that was particularly major. Nobody found anything in Rybka even though it was hundreds of ELO stronger than fruit. Yes, there were little things and some good idea, but no single idea amounted to anything that made any real difference.

Believe me when I say disassembling any program including the one you are calling your own just to find idea we can use is probably the slowest path to any kind of real success.

Thanks for your answer. If this is confirmed by the disassemblers then you will have my respect for a great achievement in computer chess. I must admit that I now use Houdini myself for analysis, as I regard it as the best available Rybka derivative.

Surely you didn't really believe that the 50+ Elo progress in Houdini 1.5 is just by some smart compiling or godly tuning, as some seem to suggest?

Since the evaluation in Rybka 4 changed quite a bit from my eval in Rybka 3, I actually find that the evals in all the Ippo family programs and Houdini are closer to my own than are the evals in Rybka 4, and so I prefer them!

Thank you for confirming that the evals from Houdini 1.5 and Rybka 4 are very different.

But first can you comment on the scaling down of the eval in Houdini? I found a version of Houdini that someone made that reverses this scaling down (I don't know how they made it) which makes it more comparable to Ippo family. Can you tell us why you downscale the final eval for output?

Houdini's evaluations are an attempt to produce a score that is consistent across the stages of the game and to make a +1.0 eval correspond to approx. 90% chance of winning the game.
The hacked Houdini versions that are now circulating may have some of this balancing code disabled, and may no longer take into account certain elements relating to endgame tablebases, contempt and the stage of the game.

Robert

[Houdini]

Even if I didn't believe Houdini started as disassembled code, it is accepted that ideas from other programs may be used, but not code or numerical tables. If someone has learned of some new ideas by disassembly and is willing to share these ideas with the community, we all benefit together. We just don't do disassembly ourselves.

It remains amazing that you're actively pushing people on this Forum to disassemble another top engine to obtain information.
Love your "we just don't do disassembly ourselves"...

I note though that Ivanhoe seems to have experienced a similar large Elo jump between version 47 and 49 in roughly the same time period, and I wonder if this is coincidence or not.

What "Elo jump" between Ivanhoe 49 and 47 are you talking about? What ratings or results is your claim based on?
Are you aware that:
A) the Ivanhoe version numbers are counting downwards;
B) most people seem to think that the more recent compile 47 is less good than the older 49; (Martin Thoresen was questioned by some for having used the "less good" version 47 in the recent TCEC Division I).
C) a study of the Ivanhoe 47/49 codes would show you that hardly anything has changed in the engine and that no Elo change is to be expected.

Since you are redefining 1.0 in this manner, it would have been desirable to state so before this, but better late than never!

Let me refer to my TalkChess post from May 16, 2010, 19:21 (the day I released Houdini 1.0), which demonstrates that I haven't redefined anything since the start:

As a guideline to Houdini, I tried to produce the most accurate evaluation as possible. In my view an evaluation of +1.00 (a clear pawn ahead) should mean that the position is nearly winning, as would be the case in most human chess matches.

Robert

[bob]

Were the bugs in Ivanhoe ones that would affect the Elo rating on a single core? I wonder whether bug fixes alone could account for the higher Houdini rating. Also, the rating jump in Houdini 1.5 corresponds in time to the addition of LMR at PV nodes in Ivanhoe. Assuming this was copied into the latest Houdini, this could account for the large speedup and much of the Elo gain. What do you think?

LMR at PV nodes is -5 Elo. Is Don really so clueless?
And you believe what Chris writes???

For us LMR at PV nodes is significant, perhaps 20 - 30 ELO but I don't remember off the top of my head. We have tried several times to turn it off and try for more aggressive LMR and other things in non-pv nodes but in each case we have failed. LMR at PV nodes is clearly a good thing.

Ivanhoe is a really good program, it would only take a few changes such as this to explain the modest improvement or the first houdini over Ivanhoe.

Anyone that thinks LMR doesn't work at PV nodes simply has a bug or some design flaw. If you think about the tree, if something works at a PV node, it should work everywhere. If it doesn't work at a PV node, it should not work anywhere. Why? Because quite often, those "non-PV" (non first moves at the root) become PV nodes. You can't afford to search them one way when they are not part of the PV, and then re-search them a different way when they threaten to become such...

One might try to apply the same logic to extensions. Or null-move pruning. And get the exact same negative impact... If you are so sure your PV is correct and nothing will change it, why do the search at all, right?

I have tested the idea. While I don't remember the exact results, it hurt somewhat. There are very few PV nodes, so it isn't huge. But it is there.

[bob]

This statement identifies you as an amateur at software development. There are bugs in EVERY substantial piece of software and of course they wouldn't be there if the programmers knew about them. Since they don't know about them, they don't know how much they hurt the program.

A programmer might make the statement that they believe with a great deal of confidence there are no substantial bugs affecting ELO, but they can never know for sure. But this would be for the programmers to say, not for you.

When ppl get to certain age they start understanding things only literally. Moreover, they also see a huge decline in their intellectual performance and they get very bitter because of that, and very jealous to young ppl obviously much more capable than what they are at the moment.
I don't approve this, but I can understand it.

Non-old, non-grumpy man regards,
Milos

"fully debugged chess engine" is a pure oxymoron. And you won't find a single programmer (emphasis on programmer here, defined as someone that has actually _written_ a chess program) that would disagree. Every time I hunt for a bug, I find another bug or two before I find the one I am looking for. And from one that _has_ been doing this kind of programming for 43 years now, nothing has changed with respect to this over those 43 years. I probably introduce fewer bugs now than 43 years ago due to experience, but I will always introduce them until I stop programming...

[paulo]

So a "man of science" should be ok with breaking into a competitor's lab and stealing his science? Or a "man of science" should be able to copy significant parts of someone else's work and publish it as their own, ignoring copyright law, ethics, etc?

I do not get the argument. Yes, it would be nice if everyone worked for the "common good". And some of us share every new idea we discover and do exactly that. And some don't. And some try to short-cut the time/effort part of the equation and simply copy and change and then say "this is my original work."

Robert,

IMO a "man of science" should be ok with breaking into a competitor's lab and stealing his science as long as that contributes to the "common good". Definitely yes. Knowledge must be shared, this is the way I see it.

BR
Paulo