First experience with Igorrit

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Uri Blass
Posts: 10268
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: First experience with Igorrit

Post by Uri Blass »

El Gringo wrote:Hi All,

Until now, i was avoiding any contact with Ippolit, robbolito, iggorit and Ivanhoe's, due to the cloning issue...

But i could not resist any longer....

So i played a little match between Rybka 3 x64 2 CPU and Igorrit 0.086v9 x64 2CPU :

Code: Select all

Igorrit_0086v9_x64 - Rybka 3 x64 2CPU : 18,0/30 8-2-20 (======11=10===1==1=0=11====1==)  60%   +70
Hardware : E5200 @3.7Ghz, 40moves/10 minutes, Arena GUI + Arena mainbook 10 moves.
+70 Elo ! Never saw a clone engine that is so much stronger than the original ?!?

Best
Johan
8-2 and 20 draws can also happen between engine and itself and it certainly is no proof for +70 elo.

You also cannot decide about elo improvement only based on testing against previous version.

Uri
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: First experience with Igorrit

Post by yanquis1972 »

in 50 games at 2+6 with a previous IG compile, i got +80 elo

with firebird in 15+10 after 37 games i am currently getting +67 elo.

so i think there is something to the idea that the MP clones are stronger.
User avatar
M ANSARI
Posts: 3707
Joined: Thu Mar 16, 2006 7:10 pm

Re: First experience with Igorrit

Post by M ANSARI »

Was this testing done using R3 on 0 contempt? Although I haven't tested on 4 cores, on 8 cores R3 is around 50 ELO stronger. I do use contempt 0 though, which is the default analysis R3 mode. I plan to try to play R3 0 contempt against another R3 15 contempt and see what the ELO difference is.
El Gringo
Posts: 118
Joined: Tue Nov 27, 2007 4:01 pm

Re: First experience with Igorrit

Post by El Gringo »

wolfv wrote:Dear Johan,


my "old" team-mate in computer chess and our two-time trophy bearer in Leiden :-). It's only every now and then that I take a peek at CCC and after seeing your message it was I who couldn't resist to say hi and wish you all the best in 2010 :-)

With regard to your results I should say that I have about the same margin in my testing the new Igorrit and Rybka --- at game in 10 min Igor is leading by a hefty margin, yo-yoying between +45 and +65... which is an excellent result. I play matches on my i5-750 box and it seems almost obvious after about 350 games that Rybka's finally been dethroned as the margin is at an almost steady + 60 in favour of Igorrit.

Naturally, many issues are unclear here. Someone mentioned that it was possible for a cloner to make a superior successor, e.g. Toga (supposedly a clone?) was better than Fruit, but I would like to ask if this applied to Rybka ver. 1, too -- (rising from a mere 2100 elo and one-more-uci-engine obscurity to 3000 elo in a matter of weeks after the publication of the Fruit sources). How did it come to be so much better than Fruit, but the answer for many here is at hand, they say that it's been shown rather clearly that Rybka ver. 1 had substantial parts (not only snippets, as often said) of Letouzey's program.

It is apparently a very complex story of double standards --- still, one person could eventually resolve the matter of who "stole" code from whom (Vasik Rajlich). What is needed is evidence that points to parts of code in Ippo that were stolen from Rybka. However, the waters are much too murky at the moment and Vasik's keeping them that way... doing nothing to help us all get to the truth.

From my past dabbling in chess programming I am rather convinced that Igo/Ippo/Robbo, etc. are not merly clones but vastly improved versions of some other programs. Not necessarily only Rybka...


If the vindication of Rybka ver. 1 was based on its phenomenal playing strength and the oft quoted proverbial scientific leaps achieved by standing on the shoulders of the giants preceding us, then lets give Ippo the benefit of the doubt too, using the same cliche. Unless (and until) proof of theft is offered, of course.

I said all this so as to try to convince both myself and you to feel relaxed about using the program and testing it freely as its status is "innocent until proven otherwise". If it turns out that there is clear evidence that Ippo is a clone -- then we should chuck it away and go back to Stockfish or Doch or Shredder or Fritz, or Ruffian, for that matter :-).


Good to see you here again, I was really glad when I ran across your post. Best wishes to the family and happy testing.

Your team-mate and chess friend,


Djordje

Hi Djordje,

My best wishes for 2010 too !! I'm glad to hear something from you.

Yes, we had great times testing our beloved Ruffian.

You have some good points here in the ippo/robbo/iggorit question and i can find a lot off my thaughts in your opinion too.

'Innocent until proven otherwise' is certainly true, not only for the people behind Ippo/Robbo/.... but also for Vas (the fruit story).

Best Regards
Johan
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: First experience with Igorrit

Post by yanquis1972 »

M ANSARI, i will provide a more detailed breakdown later, possibly at the rybka board or here as well, but after a set # of games (forget how many) i changed rybka contempt to 0; result was something like +3 =7 -2 in favor of firebird. after that i changed firebird pawn hash to 32MB (keepign contempt at 0 for rybka) and the score was +3 =8 -0. so overall it seemed to not make much difference. FB was pretty consistently +70 elo for the match. i may continue it very slightly to get an even 50 games, perhaps giving rybka the white pieces in all, because whenever i restarted firebird got white & it may have had a slight advantage there depending on my timing.
Albert Silver
Posts: 3019
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: First experience with Igorrit

Post by Albert Silver »

M ANSARI wrote:Was this testing done using R3 on 0 contempt? Although I haven't tested on 4 cores, on 8 cores R3 is around 50 ELO stronger. I do use contempt 0 though, which is the default analysis R3 mode. I plan to try to play R3 0 contempt against another R3 15 contempt and see what the ELO difference is.
I could be wrong, but my testing has consistently shown Contempt = 0 to be a weaker setting.

That said, I am testing Robber0.09x64 against R3x64 using the Silver Suite at 40/40 as I was curious to test the recent very surprising info that the difference in Blitz levels out enormously at slower TCs. 40/40 would be considered a slow TC by all means, and with SP, I am also testing with ponder on as well. In theory, other than the slower TC, this should be as advantageous as can be for Robber, since my previous g/10 testing showed a significantly greater edge for Robber with ponder on than without.

Code: Select all

Robbog3x64-r3x64SP-ponderon-silverfull  2009

                           
1   RobboLito 0.085g3 x64  +76 Elo  +17/=34/-5 60.71%   34.0/56
2   Rybka 3                -76 Elo  +5/=34/-17 39.29%   22.0/56

Code: Select all

Robbog3x64-R3x64SP-ponderoff-silverfull  2009

                           
1   Rybka 3                -38 Elo  +28/=55/-17 55.50%   55.5/100
2   RobboLito 0.085g3 x64  +38 Elo  +17/=55/-28 44.50%   44.5/100
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: First experience with Igorrit

Post by lkaufman »

Based on a huge sample (150k+) of positions from top-level correspondence play, I compared Rybka 3 on one core to Robbolito. At one second per position, Robbolito matched the move choices better than R3 by nearly one percentage point, roughly 50 Elo, in line with your results. However at one minute per position (using a sampling of 3k positions), Rybka scored better than Robbolito by an even larger margin. So this suggests that while Robbolito is surely stronger in blitz, it may be weaker at 40/2 levels or thereabouts. Of course it would take a lot of games to prove this either way. My suspicion is that Robbolito has merely tweaked various parameters (and omitted eval terms) to perform better at blitz at the expense of results at slower time controls. It's premature to say that Robbolito (or its kin) is an actual improvement over Rybka at serious levels.
BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 5:18 am

Re: First experience with Igorrit

Post by BubbaTough »

lkaufman wrote:Based on a huge sample (150k+) of positions from top-level correspondence play, I compared Rybka 3 on one core to Robbolito. At one second per position, Robbolito matched the move choices better than R3 by nearly one percentage point, roughly 50 Elo, in line with your results. However at one minute per position (using a sampling of 3k positions), Rybka scored better than Robbolito by an even larger margin. So this suggests that while Robbolito is surely stronger in blitz, it may be weaker at 40/2 levels or thereabouts. Of course it would take a lot of games to prove this either way. My suspicion is that Robbolito has merely tweaked various parameters (and omitted eval terms) to perform better at blitz at the expense of results at slower time controls. It's premature to say that Robbolito (or its kin) is an actual improvement over Rybka at serious levels.
Is this list of correspondence game positions publicly available, or is it a private collection?

-Sam
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: First experience with Igorrit

Post by lkaufman »

private (for Doch), at least so far.
beram
Posts: 1187
Joined: Wed Jan 06, 2010 3:11 pm

Re: First experience with Igorrit

Post by beram »

lkaufman wrote:...... My suspicion is that Robbolito has merely tweaked various parameters (and omitted eval terms) to perform better at blitz at the expense of results at slower time controls. It's premature to say that Robbolito (or its kin) is an actual improvement over Rybka at serious levels.
Please play testmatch for yourself, I am convinced:

http://www.talkchess.com/forum/viewtopi ... 554939a35d

And yet at longer time control an even better performance:

Compaq 6710b, Blitz 25m, Nunn2 Test Match T8300 2100 Mhz, 64 Mb Hash

1 FireBird 1.0 beta w32 +16/=28/-6 60.00% 30.0/50
2 Rybka 3 32-bit +6/=28/-16 40.00% 20.0/50

Bram Mourik