Allow steroids

[benstoker]

It took Vas two and a half years to improve Rybka for less than 40 Elo. Allegedly his own engine. And you think he could improve Ippo for 60-70 Elo as Robert did in six months???

The fact that you continue to stress this, suggest you have really no idea as to how stupid a statement that is.

Of course Vas cannot easily improve Rybka. Because it is his own program! If he could improve it, he would already have done so. He put every trick in there he knows, so he is out of tricks.

But other top programmers know other tricks, that Vas has never heard of (like they don't know all Vas' tricks). By adding the tricks they know, and Vas didn't, they can quite easily improve it. While Vas himself could never add the tricks he didn't know.

Your statement just reveals that you have no clue at all what is involved in the development of a Chess engine.

I agree with you HGM. Let's just completely do away with that stupid ELO crap. Who cares? It proves nothing. And besides, you have to run over 30,000 games before you get a statistically relevant number. Who has the friggin' time for that? HGM, why do we even bother having our chess engines play to a win, lose, or draw? For God's sake, who cares? I know this much, HGM, if I know anything at all: I know I don't need to know if my engine can beat some other engine to know that my engine is original. And for God's sake, that's all I care about. Screw this b.s. win, lose, draw crap. That's for the OTB woodpushers.

[SuneF]

Let me quote you Bob on this:

The author decided to throw R4 not with the optimal strength, because it would be stlen (decompiled) by your allies. I read that already bugs weaken a program. Now the cluster Rybka gets full strength. What's your comment?

A succinct one; "bullshit".

Needless to say most of the reasonable ppl would agree with Bob's comment...

Hmm I doubt that.
It wouldn't make any sense for a guy like Vas to release a new engine with all his best tricks knowing full well it would probably get reverse engineered again!
Fool me once, shame on you, fool me twice, shame on me, as they say.

This is a pretty interesting problem actually. I wonder if we will ever see new strong Shredders and Fritzes or whether they have finally realized that this market is ruined.
Put yourself in their shoes for a moment, imagine that you work really hard for 6 years, put in years of cpu time and finally achieve your goal of having the strongest engine. Now just when you are right there where you've always wanted to be it gets reverse engineered and you lose it all!!

Next week some *** comes in the door, "hey look guys, I just wrote a new super engine, I am a genious programmer, it is 20 Elo stronger than #2 and I am giving it away for free. Commence your worshipping at once, I am the greatest."

Not sure whether to laugh or cry, it's a tragic bad joke.

And let me give you a bit of secret. Houdini improvement in strength is definitively not in eval .

You totally missed the point of his post. Did you even bother to read it?

[hgm]

I agree with you HGM. Let's just completely do away with that stupid ELO crap. Who cares? It proves nothing. And besides, you have to run over 30,000 games before you get a statistically relevant number. Who has the friggin' time for that? HGM, why do we even bother having our chess engines play to a win, lose, or draw? For God's sake, who cares? I know this much, HGM, if I know anything at all: I know I don't need to know if my engine can beat some other engine to know that my engine is original. And for God's sake, that's all I care about. Screw this b.s. win, lose, draw crap. That's for the OTB woodpushers.

It sounds a bit like your brain just short-circuited. Too bad, just after a previous post showed you had finally gotten the point. How is any of the nonsense you write above "agreeing with me"? Or was that supposed to be the biggest nonsense of all?

[hgm]

Then, let´s make a compromise solution. Let´s create two tournaments categories:

1. "Original Engines" Tournaments

2. Absolute Strength Tournaments.

I would not have anything against that. The problem is that the people that like style (2) tournaments (or at least the most vocal of that group) do not seem to accept this solution. They seem to be set to eradicate style (1) tournaments, rather than putting their effort in enjoying style (2) or seeing that there are more of those. The original reaction of Ben when I pointed out style (2) contests existed, (at playchess, and I could also mention TCEC) and that he could there to play / watch / cheer was not met by the reaction "Great, I will go there at once" or "Good, I will organize one more myself", but in stead was met by "but then we would have to disband every style (1) tournament"...

Of course organizers and participants of current style (1) tourneys are quite happy with the way their tournaments are doing, and style (2) is not very interesting to them, so they are unlikely to show up there. No amount of talk or babble on this forum will ever change that, much to the chagrin of those who want to bend the world to their will.

[Milos]

I would not have anything against that. The problem is that the people that like style (2) tournaments (or at least the most vocal of that group) do not seem to accept this solution. They seem to be set to eradicate style (1) tournaments, rather than putting their effort in enjoying style (2) or seeing that there are more of those. The original reaction of Ben when I pointed out style (2) contests existed, (at playchess, and I could also mention TCEC) and that he could there to play / watch / cheer was not met by the reaction "Great, I will go there at once" or "Good, I will organize one more myself", but in stead was met by "but then we would have to disband every style (1) tournament"...

Of course organizers and participants of current style (1) tourneys are quite happy with the way their tournaments are doing, and style (2) is not very interesting to them, so they are unlikely to show up there. No amount of talk or babble on this forum will ever change that, much to the chagrin of those who want to bend the world to their will.

One of the rare posts of yours I agree completely.
The problem I see is when (1) tourneys are presented as (2) tourneys.

[Don]

I get the feeling that you keep trying to steer away from the actual issue here. I don't have a disagreement that Houdini is the strongest program or that the author make big improvements to the base code. We seem to agree on this.

My bone of contention is the dishonesty.

What is rather humorous is that even YOU accept that it's a clone and have spent a good deal of effort justifying it and explaining the reasons why you feel it's ok and the nothing wrong was done in starting from Ivanhoe. And the yet the so called author of Houdini vehemently denies that any of this was done! It's funny!

You sympathize with him and it doesn't bother you that he is being dishonest, so you are being complicit.

It's like the Bush war - everyone who really wanted the war argued that it was justified despite overwhelming evidence to the contrary and just like Bush I think most of the population who wanted it knew it was all nonsense, but they wanted it. Or the mobsters of the mid century who people secretly admired and winked at.

Don

If I take Houdini and binary edit it my own name, then I have made a wonderfully original program, because it is at least as strong as the strongest program in the world.

You think? Is that what Houdini is, then? A mere hexedit job? Really? If you bring your hexedit job to the First Loser Tourney and only come in as strong and not stronger than some other hexedit job, then no, I have no idea beyond looking at stdout whether your hack is original or not. But, I'll say it again and again, if your hexedit hack, for reasons known only to God, manages to trounce all other engines in a significant way (as Houdini has), then, I gotta hand it to you, brother, you got to be some kind of genius having created Top Dog with nothing more than a hex editor. That is freaking Original with a capital O.

This is your misconception and you are wrong about it. The analogy which is most appropriate is the marathon analogy.

Please, I think the analogies are, shall I dare say it, getting very anal.

Every reasonable chess programmer (who uses good testing methodology and is reasonable talented) can improve on Ivanhoe. For example if the Stockfish team started with Ivanhoe I am positive they would find some substantial improvements. So could Gian-Carlo, Vas, and ANY of the good engineers in computer chess.

And I certainly can see and appreciate how a serious engineer who takes pride in his own work would have little interest in simply fiddling with IvanHoe as a starting point. Though, I'm sure there are other respected engineers who would not recoil at the thought.

I wonder, what if somebody extracted some of the good stuff out of IvanHoe and submitted it for inclusion in Stockfish, saying up front "hey guys, I got this out of IvanHoe ..."? Should the Stockfish gang reject it?

We basically use a trial and error procedure. ALL of us have good ideas and we implement them and then test them. The ones that don't work we discard and the ones that work we keep. When we look at other peoples source code (which we do) we see things they did wrong - and if they looked at ours they would see things we did wrong. I could improve Stockfish, Houdini, Ivanhoe or any program you could give me and I'm quite sure all the top guys could do so too. As a matter of fact, look how quickly the already brilliant program Glaurung improved when the talents of others was added to the team.

However, improving a very strong program is not the same as MAKING a very strong program from scratch. When you start from someone else's best work, you start with all of the THEIR good ideas then you add your own to it. If you start with the very best program you have basically pushed your way to the front of the line and you take off from there.

I think there may be some kind of bifurcation of standards between the open source crowd and the closed source crowd. And I didn't realize that or fully appreciate it. After all, with open source, it is only expected that I would stand on the shoulders of giants. With open source I do get to jump in at the middle of the race and there's no problem with that. That's the whole idea of open source. Take other's work, improve it, contribute your improvements back. That goes on here in talkchess and at open-chess, where code for Stockfish is contributed all the time. I think it's done a lot in the open source world. Apple's Safari originated from the KDE Konqueror browser. They took it, improved it, renamed it, and, if I'm not mistaken, eventually contributed some of the code back - but I don't remember - they should have, if they didn't.

With open source, I can't fake it. Hex editors will get me nowhere and are pointless. With open source, I have to write the code and it's useless if it's not original and does not improve the program. With open source, people really do get to look at what I did. And if I have come up with a new idea that improves the science of computer chess, that idea is expressed openly and subject to scrutiny and peer review. That idea can be tested by others to see if results can be duplicated, and thereby confirming the advancement of the science.

Closed source is just different. Closed source is just a completely different game, isn't it? There's no community contribution. Indeed, code inspection is not just verboten, it becomes almost illegal. Closed source involves just one man or a close company of individuals working privately. How am I suppose to know when or if something original has been contributed to a closed source program? It's a black box.

Say you give me two closed source programs. You ask me to tell you if they are different. What can I do? I guess I'd first note the file size. I can whip out a hex editor. I'd could call up Dr. Hyatt and beg him to run these things on his cluster and tell me which one plays better. If there was a difference, and one was stronger than the other, then I'd know that much. What else can I do to examine a closed source prog? I'd look at its stdout output. Compare that. What else? Send it to BB and Zach and beg them to do some disassembly judo on it. What else? Ask the author? That's probably the last thing I'd do. Like the old rap song, "ya cant truss it."

Houdart dumps his closed source program on the world. It's another black box. If he's done nothing other than copy IvanHoe code and renamed it, then he's a joke. I cannot see the code. What do I do? Well, the file size is different. Somehow I manage to talk Dr. Hyatt into testing it. He reports back to me that Houdini crushes everything. I ask him, "how? what's new about it." He says, "Hell if I know. I got better things to do. Ask Zach." Zach calls BB and they spend a couple months doing their disassembly judo on it. They issue a report. The conclusion is expressed in words like this: "there are enough similarities to say that Houdini is in some areas substantially similar, if not identical, to IvanHoe and in other parts substantially dissimilar." Thank you BB and Zach. After all that, I still don't really know much about Houdini, other than the glaring fact that it is significantly better than everything else.

And so although Houdini is clearly original in some significant respect, I can never know why or how. It's the best engine in the world that does not advance the science of computer chess one bit, since the knowledge of its improvement is private and secret.

Closed source engines are never in a race of Ideas. That's not their game. The Ideas expressed in closed source engines are unknown, or are only vaguely hinted at by their mealy-mouthed authors. I can go to the First Loser Tourney with my Hexeditorolito closed source engine and intone to the slobbering crowd that it employs my new Super Duper 4th Dimension Bitboards idea. That's nothing more than a stupid advertising slogan. It may not even be true. In fact, truth be told, I made it up when I was drunk. It doesn't even matter! Who cares? Because if Hexeditorolito trounces the competition, I go home with the prize regardless. Whatever I did to make Hexeditorolito win, nobody will ever know! Ain't that great? All I hope is that I can sell enough Hexeditorolitos before those annoying holier-than-thou disassemblers are on to me. Sound familiar?

Which is why ... closed source engines are only in it to win, to beat the other engines. The closed source engines can't say their race is about Ideas of computer chess ... ever ... UNLESS the ideas are EXPRESSED. Fabian's Ideas were expressed when he opened the source. And the science was advanced.

If any closed source author proclaims he's written an engine from the ground up that is all his, I really have no reason to believe it. My response is SHOW ME THE MONEY. Because, with a closed source engine, what the hell do I care what Ideas are inside the engine -- the author ain't going to tell me. All I care about is whether the closed source engine performs. Did it win or lose. Is it Top Dog or First Loser.

It does take a certain amount of talent to improve a very strong program, but you are completely ignorant about the process if you believe that is proof of genius. Houdini is a case in point. The day Robert Houdart put the sources to Ivanhoe on his hard disk and put his name on it, he immediately joined the ranks of the elite in chess programming. There is a discussion on the Ippolit forum about the changes he made to obfuscate what he had done, but the bottom line is that he STARTED from that point and then moved forward from there. So for as long as he didn't get caught, he had the best program in the world.

A more accurate way to characterize this is to say that Houdart helped build the best program in the world, but he only helped. He does not deserve top billing on the marquee, but people like you keep him in business.

Agreed. Houdart did something, who knows what, that deserves credit (that's what I'm saying). Agreed. He didn't do it on his own from ground up. And I would add he deserves discredit for taking an open source program and making it closed source. Whatever else is said, it's the best program. It's original in some significant respect, and it cannot be any more a pariah than any other closed source black box chess engine.

[Don]

Let me quote you Bob on this:

The author decided to throw R4 not with the optimal strength, because it would be stlen (decompiled) by your allies. I read that already bugs weaken a program. Now the cluster Rybka gets full strength. What's your comment?

A succinct one; "bullshit".

Needless to say most of the reasonable ppl would agree with Bob's comment...

Hmm I doubt that.

I think there is a lot of room for disagreement on this. I personally think it's likely that Vas held something back, but Bob could be right on this too. If I had to assign a probability to this it would be something like 70% that he held back. But if I found that it was his best, I would not be surprised either.

It wouldn't make any sense for a guy like Vas to release a new engine with all his best tricks knowing full well it would probably get reverse engineered again!
Fool me once, shame on you, fool me twice, shame on me, as they say.

This is a pretty interesting problem actually. I wonder if we will ever see new strong Shredders and Fritzes or whether they have finally realized that this market is ruined.
Put yourself in their shoes for a moment, imagine that you work really hard for 6 years, put in years of cpu time and finally achieve your goal of having the strongest engine. Now just when you are right there where you've always wanted to be it gets reverse engineered and you lose it all!!

Next week some *** comes in the door, "hey look guys, I just wrote a new super engine, I am a genious programmer, it is 20 Elo stronger than #2 and I am giving it away for free. Commence your worshipping at once, I am the greatest."

Not sure whether to laugh or cry, it's a tragic bad joke.

And let me give you a bit of secret. Houdini improvement in strength is definitively not in eval .

You totally missed the point of his post. Did you even bother to read it?

[bob]

Let me quote you Bob on this:

The author decided to throw R4 not with the optimal strength, because it would be stlen (decompiled) by your allies. I read that already bugs weaken a program. Now the cluster Rybka gets full strength. What's your comment?

A succinct one; "bullshit".

Needless to say most of the reasonable ppl would agree with Bob's comment...

Hmm I doubt that.

I think there is a lot of room for disagreement on this. I personally think it's likely that Vas held something back, but Bob could be right on this too. If I had to assign a probability to this it would be something like 70% that he held back. But if I found that it was his best, I would not be surprised either.

It wouldn't make any sense for a guy like Vas to release a new engine with all his best tricks knowing full well it would probably get reverse engineered again!
Fool me once, shame on you, fool me twice, shame on me, as they say.

This is a pretty interesting problem actually. I wonder if we will ever see new strong Shredders and Fritzes or whether they have finally realized that this market is ruined.
Put yourself in their shoes for a moment, imagine that you work really hard for 6 years, put in years of cpu time and finally achieve your goal of having the strongest engine. Now just when you are right there where you've always wanted to be it gets reverse engineered and you lose it all!!

Next week some *** comes in the door, "hey look guys, I just wrote a new super engine, I am a genious programmer, it is 20 Elo stronger than #2 and I am giving it away for free. Commence your worshipping at once, I am the greatest."

Not sure whether to laugh or cry, it's a tragic bad joke.

And let me give you a bit of secret. Houdini improvement in strength is definitively not in eval .

You totally missed the point of his post. Did you even bother to read it?

Obviously, with Vas, anything is possible, based on past actions. But holding back something on a new release? That is the kind of thing that can backfire and lose you a lot of business.

I get a bit aggravated at the video game market. For example, one favorite of mine is Call of Duty (black ops). Sells for 60 bucks and up depending on which version you buy. Then 3 months later, they add 4 new maps for about 15 bucks. Then 3 months later... And if you don't get the new maps, online play becomes problematic. I'd rather be told "OK, 120 bucks will get the current version + 4 map packs, one every 3 months." That's honesty. I'd be very unhappy if I bought a new version (say black ops after playing every other version from Call of duty 4 onward) and found that black ops offered very little new material over the last modern warefare 2. When one spends money, one expects to obtain some value for that money. To hold part back so that you can then release (and charge for) an update in a few months is borderline dishonest.

[Don]

Obviously, with Vas, anything is possible, based on past actions. But holding back something on a new release? That is the kind of thing that can backfire and lose you a lot of business.

I am just guessing here, but I think he was committed legally to providing an upgrade but his real interest was in cluster Rybka. Of course I might be totally wrong about that, but it's true, it seems likely that he might have held something back. I had heard that he indeed planned to hold it back but I don't remember where I hear that and it might just be a rumor.

I get a bit aggravated at the video game market. For example, one favorite of mine is Call of Duty (black ops). Sells for 60 bucks and up depending on which version you buy. Then 3 months later, they add 4 new maps for about 15 bucks. Then 3 months later... And if you don't get the new maps, online play becomes problematic. I'd rather be told "OK, 120 bucks will get the current version + 4 map packs, one every 3 months." That's honesty. I'd be very unhappy if I bought a new version (say black ops after playing every other version from Call of duty 4 onward) and found that black ops offered very little new material over the last modern warefare 2. When one spends money, one expects to obtain some value for that money. To hold part back so that you can then release (and charge for) an update in a few months is borderline dishonest.

Isn't it often the case with processor chips that that different versions of the same chip have nothing missing over the other versions, but it's just disabled? I know that some of the old pentiums had the floating point circuits, but the less expensive version just had this disabled.

I suppose you could do that with chess programs. You could have a 3300 ELO program with various enhancement disabled. (Komodo has dead code lurking in there, but it's usually not something that helps the program and it's probably compiled away.)

Don

[michiguel]

It did not matter much to the IOC that Ben Johnson ran the 100m dash faster than any human has ever done.

Nitpick: Faster than any human had ever done. Four men (Usain Bolt, Tyson Gay, Asafa Powell and Nesta Carter) have run faster now. Two more (Justin Gatlin and Tim Montgomery) also had faster times, but were later disqualified for doping offences.

They took away his gold medal and removed him from the ranking, and suspended him for his useful life! And there is very wide consensus that this was the right thing to do.

Of course it was. I find the example of Ben Johnson quite relevant to this discussion, because the supporters of derivative engines like to make the argument that the "authors" of the derivative engines must be very talented, because the derivatives are far stronger than the original. Perhaps some of them are indeed very talented, but so was Ben Johnson. Far more talented in his field, in fact, than any chess programmer in their field since Alan Turing. But extraordinary talent does not excuse cheating.

Moreover, starting with a strong, stable chess engine is an incomparably bigger advantage in computer chess than using steroids is in sprinting. Steroids don't turn average athletes into world-class sprinters. They may give you 0.1 seconds or so on the 100 meters, which could be enough to put you ahead of your rivals if you are already an elite sprinter, but it won't help you much if you're not already there. Using an existing top chess engine as the basis for your own program, on the other hand, allows you to jump directly into the world elite with no effort. It isn't equivalent to using steroids in sports. It's equivalent to magically swapping your body for Ben Johnson's body at his prime, and to start training for the 100 meters from there.

I sometimes do feel a little sorry for Ben Johnson, though. He is always the example that is brought up, despite having competed in a period were steroid use was even more widespread than today. Nobody seems to remember that half of his rivals also got caught.

I suspect that it was institutionalized and in practice by each and every nation that participated in the olympics to some degree at least. Similarly, with American baseball, I think that steroids were nearly forced upon the players by the atmosphere that was prevalent. There is a report with some huge number of baseball players that got caught. I guess that the number of users would make it the tip of the iceberg.

Cheating (and acceptance of cheating) are symptoms of the general dishonesty we find in society today. I find it rather disturbing.

That's right. Acceptance of cheating is even worse than cheating.
Miguel

"History will have to record that the greatest tragedy of this period of social transition was not the strident clamor of the bad people, but the appalling silence of the good people."
Martin Luther King, Jr.