TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Tom Barrister
Posts: 227
Joined: Tue Oct 05, 2010 3:29 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by Tom Barrister » Tue Nov 30, 2010 10:05 pm

Don wrote:Tom,

The only thing that matters is how close you are to the top program. If you just measure improvement of each individual program it is based on how crappy it was at the point you start measuring.

There was a point in time when Rybka 3 first came out that there was a huge gap in performance between him and anyone else.

You picked Rybka 2 as your starting point.
Actually, I picked a point in time in which all of the main competitors of that era were present on the list and compared those with the top competitors of the current list, using Spike as an anchor.

I don't know how large a gap in ELO Rybka 3 has at a given point in time over all of its competitors, but I doubt that it was consistently huge over a reasonable stretch of time.

----
I suppose that if we look at enough lists, we'll find a large enough discrepancy at some point in time. I can point to a few lists that show Rybka over 150 ELO ahead of the next engine. I also can't find several of its closest competitors on it (and I'm going back to pre-Ip*).

Rybka is 300 points better than Fruit, which hasn't been worked on since when: early 2007? It isn't 300 ELO better than Fruit's continuation/fork of Toga. Several other engines are also much stronger than the 2007 Fruit is. Things improve. Products and/or services that remain static will usually fall behind; this has been true since long before computers came into widespread existence. One only needs to look at A&P, Montgomery Ward (and Sears, for that matter), Penn Central, Equity Corporation, General Motors, etc. to see that one can't rest on one's laurels. Each of the above was at one time the leader in its field; Penn Central and General Motors were at one time each the largest corporations on the planet. Each got complacent, and each was eaten up by more eager and innovative competitors. But I'm straying off topic.

Even if we concede that Mr. Rajlich has advanced chess engines to some degree, it's a stretch, in my opinion, to hail the man as the end-all of chess programming in the past five years. If we were to do this, I would be one of the "end-all's" in Othello programing. Many years ago, I suggested something to Mic Buro, the author of what was then one of the many strong Othello programs in competition: Logistello. It had nothing to do with programming, as I know little about that subject. It involved a certain aspect of pattern evaluation, a certain type of piece configuration that's favorable to the side who achieves it. Mic incorporated it into Logistello. During the next year, Logistello swamped the competition in most tournaments. About a year from the initial date, I suggested the same pattern evaluation to other programmers, in an off-the-cuff conversation on the IOS, where most of them hung out (had a connection open) during the weekdays. I got two or three questions and answered them. A couple said that my idea made a lot of sense. Within a month or so, four or five programs got better and became competitive with Logistello. Did my idea cause that or at least stimulate further development? Possibly, perhaps probably. Does that make me the "end all" in Othello computer programming? Hardly. I had an idea, one that most top players intuitively know about but probably don't think of in so many words, and I translated it into an idea that could be programmed into patterns that a computer could recognize.

I suppose it's just me, but I don't consider 100 ELO at a few points in time to equate blowing away the competition. It's an achievement, regardless of how it was obtained, but it isn't earth shaking, in my opinion, and it hasn't stayed constant throughout the years.

That's about all I can think of to say. I've spent way too much time on this one topic and in this one thread, not that I'm complaining, as I like to write, but I do have my day job, as do most of us, and I need to get back to it. Thanks for your attention.
This production is being brought to you by Rybka: "The engine made from scratch.™"

overtond
Posts: 22
Joined: Fri Oct 08, 2010 5:10 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by overtond » Tue Nov 30, 2010 10:24 pm

Also worth pointing out (we sometimes forget this when chasing just elo) that some of the moves made by Rybka 4 (and some other top engines) are really not that great. R4 has a number of evaluation problems that render it fairly useless as an analysis tool in many end-game positions - especially when Bishops are involved. H1.03a is certainly better - test C0.90 to see how this compares to H1.03a in the end-game - early indications are that it is fairly close between these two.

User avatar
M ANSARI
Posts: 3500
Joined: Thu Mar 16, 2006 6:10 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by M ANSARI » Wed Dec 01, 2010 6:32 am

Milos wrote:
luisrodg wrote:@Milos: "with some side help, Vincent can tell you more about it"

Please tell. I'd like to know about it as I don't.
It's the tuning thing. As you probably know Vas doesn't do any tuning himself. There was a massive automated tuning effort for R3. And I'm not talking about LK since his coefficient are nowhere near what's been implemented in R3. And as you probably know proper tuning means a lot, hundreds of elo. For example in today's standard Fruit coefficients are ridiculous. Just think of how much elo you could get from it if properly tuned (and I'm not talking about relatively amateurish efforts with Toga).
If you look at R4 it's so un-tuned, they couldn't even get TM correctly. Obviously auto-tuning resources are not available any more...
Who and why provided them in the first place, that you would have to ask Vincent (or Vas).

Milos ... now that you mention Vincent as someone you correspond with, a lot of your posts make sense. Is this the same Vincent that put out a report where he explained in great detail a major conspiracy and cover up that involved NASA the CIA and some rich arab sheikhs in making or disassembling R3? If you have anymore insider information, please share it with us! The suspense is too much to bear!

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by beram » Wed Dec 01, 2010 6:17 pm

M ANSARI wrote:
beram wrote:
M ANSARI wrote:The settings might have affected Rybka, but if you look at a bunch of Rybka 4 losses, you will find that many of them are due to not giving a drop in evaluation for blocked bishops or "bad" bishops. As for Ivanhoe and Houdini evaluating differently than Rybka ... they simply don't and evaluate almost identical to Rybka 3. Of course they have some changes and modifications, but essentially they are all based on Rybka 3 with some improvements and bug fixes. The biggest difference I can see is that they are thread based engines rather than process based.
Dear M. Ansari, you haven't read or don't agree on BB+ report published by Zach Wegner ?
http://www.open-chess.org/viewtopic.php ... =110#p1220
Respectable programmers as Ed Schroder and Chris Wittington have said that this report proves that Ippolit is not a Rybka clone

to quote BB:
..Ironically, it was M ANSARI who kept on saying (essentially) that when I dug deeper into it I would find more evidence of cloning...

I think you should go back to the Rybka forum and digg up some of the the posts I had with BB. At the time the question was if the Ippolit engines were original or not, and by original I mean that they started as code without ANY Rybka 3 RE. I had tested it with R3 and come to the conclusion that there is absolutely no way that the Ippolit engines were original and that the engine HAD to be based on R3 code. The Ippolit engines were weak in exactly the same positions as was Rybka 3 and would lose games to weaker engines in exactly the same way. Others (many posting in this very thread) were trying to convince the world that Ippolit had NOTHING to do with R3 and it was totally clean original work. I think what BB and Zach would emphatically tell you is that Ippolit is 100% for sure based on R3 code ... and if anything they put to rest ridiculous attempts to show that Ippolit had nothing to do with R3. As a matter of fact I think Zach mentioned that Ippolit gets its strength from R3 and BB clearly points out that Ippolit is RE R3. The relation that Ippolit is RE R3 is no longer in contention, but there is differing conclusions as to whether legal or ethical lines were crossed in the RE of R3. So if anything BB has shown that by digging deep into the issue, there is no doubt that the Ippolit source was derived from R3 ... now the issue is if it was legally or illegally derived ... for that to be cleared up I think Vas would have to bring out a lawsuit against the author or authors of Ippolit ... which would be difficult as they are anonymous (maybe for good reason).

So if your intention of using BB's quote was to disprove the fact that Ippolit engines were cloned or derived from R3 (which was my assertion) then you are completely wrong, as he clearly states that he thinks they were cloned from R3 ... it is just that he thinks that "cloning" or "RE" or "deriving" code ... where the code is not identical ... is IHHO fair game.

Now with regards to the continuous controversy about Rybka being a Fruit clone ... this again has gone endlessly and for some reason nobody seems to note that the issue was regarding Rybka 1.0beta which was a free engine and which Fabien himself considered re-written and clean (after looking at Strelka). I think BB himself mentioned that he would look at Zach's report and seemed to disagree with parts of it and have yet to hear him say that he thought Rybka 1.0 beta was a Fruit clone. But anyway that was Rybka 1.0 beta ... and here we still are talking as if Rybka 3 was a clone of Fruit. I have yet to see one single person say that Rybka 3 has anything to do with Fruit ... not one ... yet we have the continuous babble about ALL Rybka's being Fruit clones and thus Vas must be punished and thus cloning or stealing his work is fine and dandy ... viva la communist revolution!
Dear M Ansari,
Thx for answering. But well I was'nt able to find this in the Rybka forum. But than, look who is talking today and referring on your post now in the open chess forum ...BB+ !!
I almost was on the way to believe you, but now it furthermore looks as if you try to turn black things into blue :-)

BB+ writes: As my R3/IPPOLIT report seems to be being used in the kangaroo courts of TalkChess, perhaps I should comment:
M ANSARI: You ask "where is the proof" that they are clones ... I think the best proof is the BB report.
That's a fairly jaundiced view of the report. Maybe if "clones" were put in inverted commas I could agree. I interpret the word "clone" rather strictly, and by that measure,
R3 and IPPOLIT don't come remotely close to such a descriptor. The word "derivative" has a technical quasi-legal meaning that I prefer to avoid (similarly with "code") -- by the traditional standards of computer chess, I would say that R3/IPPOLIT and Fruit/R1 are essentially on the same footing [qualitatively, and as I say, quantitatively it can depend on your metric], in that both R1 and IPPOLIT re-use a substantial quantity of specifics of the respective pre-cursors. [The fact that Fruit was "free and open source" and R3 a "commercial product" is not relevant to me -- there are a number of dissenters in the intellectual property world, but the more common opinion is that once software is obtained legally, an end-user can use it for the purposes of discovery unless there is an agreement to the contrary][/code] :) :)

User avatar
M ANSARI
Posts: 3500
Joined: Thu Mar 16, 2006 6:10 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by M ANSARI » Thu Dec 02, 2010 6:53 am

You seem to be trying to make it look that I misquoted or tried to twist what BB had said. I certainly did not do that and you can go back to the thread you quoted and read through it yourself. I do agree that what BB considers a "clone" is not what others would consider a clone ... which is simply to copy and paste code. But that he assumes a clone to also include RE programs. I also never had an acrimonious or negative post with BB (that I can remember) and I actually did consider him as one of the good guys that was trying to get to the truth. You have to realize at the time we had one very large vocal group, trying to show in so many ways, that Ippolit was a completely clean and original engine and had NOTHING to do with R3 ... something which BB did convincingly show was total BS. At the time I simply did not think that was possible, because the engine played like Rybka 3 and analyzed like R3 and evaluated like R3 ... yes there were some tiny differences and some well documented bugs were fixed, but the overwhelming evidence was that it was cloned from R3. If I remember correctly in one of our posts I did mention that "if he would look deep enough he will find that R3 is behind it" ... which is exactly what happened. I also think I made it clear that BB did not consider the RE of Rybka 3 to be illegal or unethical ... I think he did mention that if you RE engineer a program that you legally own that you are entitled to RE it. That is his point of view and many disagree with that aspect. But the entire point of the post was that BB totally believes that Ippolit came from RE R3, and so does Zach as well as basically anyone who knows his way around chess engine code. We may all have different opinions of what constitutes a "clone" or a "derivative" ... some consider a clone to be only a true clone if the code is copied verbatim ... while others consider a clone to be a RE product. I see nothing in my post where I tried to make "something black blue".

What I do resent is the fact that BB would post in a separate forum and consider my posting to be a "kangaroo court". If there is an inaccuracy in my post the proper thing would be to point it out, preferably on the same site and thread it was posted.

beram
Posts: 1187
Joined: Wed Jan 06, 2010 2:11 pm

Re: TEST position TCEC5- Houdini 1.03a-DRybka4 1-0

Post by beram » Thu Dec 02, 2010 6:12 pm

M ANSARI wrote:You seem to be trying to make it look that I misquoted or tried to twist what BB had said. I certainly did not do that and you can go back to the thread you quoted and read through it yourself. I do agree that what BB considers a "clone" is not what others would consider a clone ... which is simply to copy and paste code. But that he assumes a clone to also include RE programs. I also never had an acrimonious or negative post with BB (that I can remember) and I actually did consider him as one of the good guys that was trying to get to the truth. You have to realize at the time we had one very large vocal group, trying to show in so many ways, that Ippolit was a completely clean and original engine and had NOTHING to do with R3 ... something which BB did convincingly show was total BS. At the time I simply did not think that was possible, because the engine played like Rybka 3 and analyzed like R3 and evaluated like R3 ... yes there were some tiny differences and some well documented bugs were fixed, but the overwhelming evidence was that it was cloned from R3. If I remember correctly in one of our posts I did mention that "if he would look deep enough he will find that R3 is behind it" ... which is exactly what happened. I also think I made it clear that BB did not consider the RE of Rybka 3 to be illegal or unethical ... I think he did mention that if you RE engineer a program that you legally own that you are entitled to RE it. That is his point of view and many disagree with that aspect. But the entire point of the post was that BB totally believes that Ippolit came from RE R3, and so does Zach as well as basically anyone who knows his way around chess engine code. We may all have different opinions of what constitutes a "clone" or a "derivative" ... some consider a clone to be only a true clone if the code is copied verbatim ... while others consider a clone to be a RE product. I see nothing in my post where I tried to make "something black blue".

What I do resent is the fact that BB would post in a separate forum and consider my posting to be a "kangaroo court". If there is an inaccuracy in my post the proper thing would be to point it out, preferably on the same site and thread it was posted.
Thx, for answering in this calm and sincere way

Then for now please let us stop these senseless discussions and - between the line accusations - about how originally these progs were made. The main matter is how good they play.

kind regards Bram

Post Reply