Robodini 1.1 300 games.

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Lavir
Posts: 263
Joined: Sun Oct 28, 2012 11:45 am

Re: Robodini 1.1 300 games.

Post by Lavir »

geots wrote: I am a bit surprised that you posted a thread on this train wreck.
I did those tests before it was 100% sure that Robodini was a RE of Houdini 3 and that it was done and posted by Richard. So I did those to test the similarities and naturally, as you say, it is just Houdini vs. Houdini.

As for the difference in elo on my tests, I think they had mainly to do with the different time management of Robodini. In fact at very long time controls (especially with ponder) the time management between the two is different in many instances.

Btw I personally think that Richard searched every cranny to see how the code works (and so here it goes the bet fable) and did many tests therein before reverting everything to default. But naturally I know nothing on these things as many say so don't ask me why I think so.
Tomcass
Posts: 786
Joined: Sun Apr 16, 2006 9:09 pm

Re: Robodini 1.1 300 games.

Post by Tomcass »

200 ADDITIONAL GAMES AT 10 MINUTES TIME CONTROL

I7 980 3.33 Ghz.
1 real core
Ponder: On.
GUI: Fritz 12
Book: Perfect 2012
Time control: 10 min + 0 sec/ game
No tablebases

201301Robodini1.1.b 2013

Robodini 1.1 - Houdini 2.0c Pro x64(x1) 28.5 - 21.5 +18/=21/-11 57.00%
Robodini 1.1 - Houdini 3 Pro x64_1 24.0 - 26.0 +16/=16/-18 48.00%

Games: http://www.mediafire.com/?6t4nk7vak13bqbc

i7 975 3.33 Ghz.
1 real core each engine
GUI: Fritz 12
Book: Fritz 12
Time control: 10 min + 0 sec/ game
Ponder: On.
No tablebases

Robodini 1.1 - Houdini 2.0c Pro x64(x1) 28.5 - 21.5 +12/=33/-5 57.00%
Games: http://www.mediafire.com/?uzwjj58prtl5yjd

Robodini 1.1 - Houdini 3 Pro x64_1 25.5 - 24.5 +9/=33/-9 51.00%
Games: http://www.mediafire.com/?7aiqqeh7hqaqx75


Overall, after 500 games:

Robodini 1.1 - Strelka 5.5 31.0 - 19.0 62.00% after 50 games

Robodini 1.1 - Houdini 2.0c Pro 110.5 - 89.5 55.25% after 200 games and

Robodini 1.1 - Houdini 3 Pro 125.5 - 124.5 50.20% after 250 games


Enjoy your week-end. :wink:

Tom.
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: Robodini 1.1 300 games.

Post by geots »

Lavir wrote:
geots wrote: I am a bit surprised that you posted a thread on this train wreck.
I did those tests before it was 100% sure that Robodini was a RE of Houdini 3 and that it was done and posted by Richard. So I did those to test the similarities and naturally, as you say, it is just Houdini vs. Houdini.

As for the difference in elo on my tests, I think they had mainly to do with the different time management of Robodini. In fact at very long time controls (especially with ponder) the time management between the two is different in many instances.

Btw I personally think that Richard searched every cranny to see how the code works (and so here it goes the bet fable) and did many tests therein before reverting everything to default. But naturally I know nothing on these things as many say so don't ask me why I think so.





Fabio, the main thing you need to take away from this whole "whatever the hell it is", and there is no argument- it is written in stone: If you want to compare two versions of the same engine, or 2 quasi-versions- you NEVER RUN THEM AGAINST EACH OTHER! The results are as useless as attacking a bear with a toothpick. You pick maybe half a dozen opponents, and run each of them against those 2 engines in exactly the same controls. And then you compare. Anything short of that is an utter waste of time. (The only exception would be, for example, running Houdini 3 against a 3 or 4 year old version of Houdini. But why would you do that anyway- when you already know the answer.) Ignore anyone telling you different- he is a screw-up looking for a place to light.



Best,

george
Gusev
Posts: 1476
Joined: Mon Jan 28, 2013 2:51 pm

Re: Robodini 1.1 300 games.

Post by Gusev »

there is no argument- it is written in stone: If you want to compare two versions of the same engine, or 2 quasi-versions- you NEVER RUN THEM AGAINST EACH OTHER! The results are as useless as attacking a bear with a toothpick.
Well-phrased rhetoric, yet no actual explanation of WHY the results of running two versions of the same engine against each other would be useless. Please elaborate.
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: Robodini 1.1 300 games.

Post by geots »

Gusev wrote:
there is no argument- it is written in stone: If you want to compare two versions of the same engine, or 2 quasi-versions- you NEVER RUN THEM AGAINST EACH OTHER! The results are as useless as attacking a bear with a toothpick.
Well-phrased rhetoric, yet no actual explanation of WHY the results of running two versions of the same engine against each other would be useless. Please elaborate.


First I would have to know if you disagree with me, agree with me but just wonder if I do know the reason- or if you honestly don't have a clue. I make this caveat because I am not sure of your sincerity in asking, mainly because if I don't have you mistaken with someone else, you already know the answer.

And BTW, nothing I have said was to cast any dispersions on TomCass. He is a superior tester- and a friend.


Best,

gts
Gusev
Posts: 1476
Joined: Mon Jan 28, 2013 2:51 pm

Re: Robodini 1.1 300 games.

Post by Gusev »

First I would have to know if you disagree with me
I mildly disagree. I don't consider the results of running two versions of the same engine against each other entirely useless. However, if one of them edges another slightly, this cannot be taken as conclusive evidence of its being better against other engines. That is to say, you are correct that these versions may have to be run against a sufficiently large variety of other engines of comparable strength if we were to draw any definite conclusions about their relative strength. I am running one compile of Fire 2.2 against another right now, and it's quite a nuisance to me, and then this. :-)
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: Robodini 1.1 300 games.

Post by geots »

Gusev wrote:
First I would have to know if you disagree with me
I mildly disagree. I don't consider the results of running two versions of the same engine against each other entirely useless. However, if one of them edges another slightly, this cannot be taken as conclusive evidence of its being better against other engines. That is to say, you are correct that these versions may have to be run against a sufficiently large variety of other engines of comparable strength if we were to draw any definite conclusions about their relative strength. I am running one compile of Fire 2.2 against another right now, and it's quite a nuisance to me, and then this. :-)






What I do not understand about all this sudden Fire interest, I ran the strongest one out there in my freeware RR tournament that just concluded. And I did not come up with that observation. I wanted to run the strongest, and Norman (kranium), who has become a good friend- hand-picked the one he preferred I run: "Fire 2.2b xTreme GH x64", George's compile. In fact when he sent it to me back last fall- there was not another like it. Because he added the "b" to make it 2.2b when he solved some split depth problems it had that was hampering its strength. Since it is no longer in active development, I just doubt he passed the copy around that had the split depth problem solved. I would not be surprised if I was the only one with it.

If you have kranium email me and tell me it is ok, I will upload it and put a link here for anyone who wants it. But my point, until I got off track as I often do- was that the Fire 2.2 xTreme GH x64, even without the "b", as far as I can tell has not been mentioned in any of the "Fire" threads. That just makes no sense, because even without the "fix" it is as strong as most all others- or stronger.

In closing, I have no problem at all with you disagreeing with me about playing 2 ver. of the same engine ag. each other. You certainly have a right to your opinions and you have the right to openly state them. In fact, we don't have enough people like you. You don't blindly follow what someone says, even if he is a "so-called" or "quasi" expert. You ask questions, and look at both sides of the issue before you make your decision. And you respect those who don't agree with you. And for that you have all my respect.



Best,

george


PS: After looking again at the Fire threads, I see there is a 2.2 xTreme x64. It may very well be GH (George's) compile and you just left the "GH" off. But at any rate- I don't see the "fixed 2.2b". If I get an ok from Norman, as I said- I will be more than happy to share it with you. It is 4:AM and I am heading to get some sleep. I shall return later.
User avatar
geots
Posts: 4790
Joined: Sat Mar 11, 2006 12:42 am

Re: Robodini 1.1 300 games.

Post by geots »

I need my stupid ass kicked good! I missed the thread where I see you do have 2.2b! And I called myself looking at the threads good. So that solves that, as you certainly don't need what you already have. I should have known since it was you, you would have all your "i's dotted" and your "t's crossed". My apologies.



Best,

george
Gusev
Posts: 1476
Joined: Mon Jan 28, 2013 2:51 pm

Re: Robodini 1.1 300 games.

Post by Gusev »

George, no problem!

Just to make things abundantly clear, for my compile I used the code hosted at http://www.chessengine.co.in/firebird_chess_engine.html under the name Fire_22_xTreme_source.rar. The 32-bit version compiled under Microsoft Visual Studio 2010 Pro just like that. For the 64-bit version, I made sure that software popcnt is utilized, because not every 64-bit machine has hardware popcnt. (Mine does, so I thought that another compile might be stronger on it, but I don't see that.)

That same site offers Fire 2.2b xTreme GH x64.rar.

I have also downloaded RobboBases from http://www.chess.cygnitec.com/RobboBases/ and made sure that they loaded.

In another thread, I see Prof. Robert Hyatt advocate "incestuous" testing to some extent. Don Dailey (if I am not mistaken) gives the example of Rybka 3 that was tested "incestuously" against its earlier version, because no other rivals came close enough at the time of its development. So we can agree on limited use of such activity.
ernest
Posts: 2053
Joined: Wed Mar 08, 2006 8:30 pm

Re: Robodini 1.1 300 games.

Post by ernest »

ernest wrote:I certainly did not get such good Robodini 1.1 result against Houdini 3.
However, using Sedat's Perfect2012c book, the match was much closer 8-)

Dualcore@3Ghz, Win XP 64-bit
1 real core
Ponder: On.
GUI: Fritz 11
Book: Perfect2012c
Time control: 2'+1"
Nalimov5 (used by Houdini, and only at GUI level by Robodini)

Houdini 3 x64 Nlm 1Thread - Robodini 1.1 x64 -Ponder
+110 -94 =188 204-188 (total 392 games)
52% or +14 Elo (95% error-bar: ±26 Elo) for Houdini 3