First experience with Igorrit

[lkaufman]

Note that 25 minutes on 32 bit is about like 15 minutes on 64 bit, which is much closer to blitz than to 40/2. So these results don't tell us which is stronger at tournament levels, especially if both programs use 4 or 8 cores.

[M ANSARI]

Actually I get also results that are directly linear with time. On 4 Ghz RL is around 60 ELO stronger than R3 at 1_1 and drops to 40 ELO at 5_1 and is 7 ELO at 16_1 etc... The only time I ever saw that was with ZM II on 8 cores where overclocking actually made ZM II stronger than Rybka 2.3.2a. I come to the same conclusion as you do in that a lot of evaluation code that was removed from R3 made RL faster, which is more helpful in fast time controls and maybe additional evaluation code is useful at long time controls. Also RL has some of R3's obvious bugs fixed, so that should add to ELO. This is especially true with MP versions as R3 gets hit with the MP hash bug very often ... anywhere from 10% to 20% of the games.

I always use contempt 0 because I think that this is the default of Rybka 3. Contempt 15 was done to improve scores against weaker engines. I think leaving it on adds a factor which can help or hurt, but does not give accurate result of the actual engine. IMHO a high contempt against an engine that is as strong or stronger is counter-productive. I can see that in the games where R3 in a totally drawn position will try to push and get a win, which will then backfire.

[Matthias Gemuh]

... a lot of evaluation code that was removed from R3 made RL faster, ...

What ??? Removing some evaluation code from R3 makes another unrelated engine faster ?
You surely want people to assume something.

Matthias.

[Albert Silver]

... a lot of evaluation code that was removed from R3 made RL faster, ...

What ??? Removing some evaluation code from R3 makes another unrelated engine faster ?
You surely want people to assume something.

Matthias.

I only assume the 92nd year of the Bolshevik revolution is finally showing the fruits of its results. That is what I understood. Did you assume something else?

[beram]

I ran yet a Nunn2 test at 40/40,40/20 sofar more equal games and stil one decisive win by... Firebird, but we will see.
Btw I found an interesting post on a test of Firebird on the CSS forum by Stefan Pohl, who has also opinions on this matter, with which I very much agree.

http://forum.computerschach.de/cgi-bin/ ... l?tid=2013
Stefan Pohl Datum 2010-01-20 08:02
"..
A test of Firebird 1.0 beta x64 on 2.83GHz Quad (64-bit Vista Home Premium) against Rybka 3 mp x64, Stockfish 1.6 x64, Naum 4th
Each of 100 games with 2 '+2'' (without permanent brain, always count turns) with Noomen preset positions. In addition, Rybka and Naum on the 3.4 and 5 TBs on a fast USB stick (could access, including 512 MB of cache). Firebird has achieved the following results against the top 3:

Firebird 1.0 beta x64 - Rybka 3 mp x64: 59.5 - 40.5 (+33 -14 = 53)
Firebird 1.0 beta x64 - Stockfish 1.6 x64: 63.5 - 36.5 (+41 -14 = 45)
Firebird 1.0 beta x64 - Naum 4: 63.5 - 36.5 (+38 -11 = 51)

Average: 62.17% against an average Elozahl of 3169 (CEGT list 4 / 40). Thus an output of about 3258, so about 20 Elo before the 3rd runner Rybka

As I follow the whole Clone debate just from the sideline, I do not know from who Firebird should be cloned. At least not from his three previous opponents. Firebird is not only too strong, but also practice a different style of play. It was noticeable that Firebird more often than his three opponents chose near the end (of yes short reflection time) decided for another line, his opponents were in this respect much more stable. __This leaves me to conclude that Firebird can profit more from longer time control (or future hardware) than others could benefit_!.
Furthermore, it appeared to me that in the endgame there is still a lot to be gained. In the endgame Firebird was far superior to Rybka 3 and still a lot of games rotated one way or another a lot, and not due to the Tablebases, but in the early final phase.
Also not unimportant: Unlike Robbolit/Ippolit, Firebird 1.0 beta, not only uses all 4 cores, but also ran (for me) stable over the whole test range. For a beta not always so obvious, and in my case obviously not true for Robbolito / Ippolit. __Also, the use of multi-core seems to be quite efficient: If you connect from 4 to 3 cores, so the number of nodes is about 20-25% back, etc. .. __””

Grts Bram

[djbl]

Dear Johan,


my "old" team-mate in computer chess and our two-time trophy bearer in Leiden . It's only every now and then that I take a peek at CCC and after seeing your message it was I who couldn't resist to say hi and wish you all the best in 2010

With regard to your results I should say that I have about the same margin in my testing the new Igorrit and Rybka --- at game in 10 min Igor is leading by a hefty margin, yo-yoying between +45 and +65... which is an excellent result. I play matches on my i5-750 box and it seems almost obvious after about 350 games that Rybka's finally been dethroned as the margin is at an almost steady + 60 in favour of Igorrit.

Naturally, many issues are unclear here. Someone mentioned that it was possible for a cloner to make a superior successor, e.g. Toga (supposedly a clone?) was better than Fruit, but I would like to ask if this applied to Rybka ver. 1, too -- (rising from a mere 2100 elo and one-more-uci-engine obscurity to 3000 elo in a matter of weeks after the publication of the Fruit sources). How did it come to be so much better than Fruit, but the answer for many here is at hand, they say that it's been shown rather clearly that Rybka ver. 1 had substantial parts (not only snippets, as often said) of Letouzey's program.

It is apparently a very complex story of double standards --- still, one person could eventually resolve the matter of who "stole" code from whom (Vasik Rajlich). What is needed is evidence that points to parts of code in Ippo that were stolen from Rybka. However, the waters are much too murky at the moment and Vasik's keeping them that way... doing nothing to help us all get to the truth.

From my past dabbling in chess programming I am rather convinced that Igo/Ippo/Robbo, etc. are not merly clones but vastly improved versions of some other programs. Not necessarily only Rybka...


If the vindication of Rybka ver. 1 was based on its phenomenal playing strength and the oft quoted proverbial scientific leaps achieved by standing on the shoulders of the giants preceding us, then lets give Ippo the benefit of the doubt too, using the same cliche. Unless (and until) proof of theft is offered, of course.

I said all this so as to try to convince both myself and you to feel relaxed about using the program and testing it freely as its status is "innocent until proven otherwise". If it turns out that there is clear evidence that Ippo is a clone -- then we should chuck it away and go back to Stockfish or Doch or Shredder or Fritz, or Ruffian, for that matter .


Good to see you here again, I was really glad when I ran across your post. Best wishes to the family and happy testing.

Your team-mate and chess friend,


Djordje

well you guys can do what you like with yours but i think i'll keep hold of mine, in the name of science n'all. whatever this represents to most my interest comes first from chess, and what is being played now by the ippollit sourced engines is the finest, strongest chess that has ever been played in the history of chess, why would i want to throw that away?

[George Tsavdaris]

That said, I am testing Robber0.09x64 against R3x64 using the Silver Suite at 40/40 as I was curious to test the recent very surprising info that the difference in Blitz levels out enormously at slower TCs. 40/40 would be considered a slow TC by all means, and with SP, I am also testing with ponder on as well. In theory, other than the slower TC, this should be as advantageous as can be for Robber, since my previous g/10 testing showed a significantly greater edge for Robber with ponder on than without.

Code: Select all

Robbog3x64-r3x64SP-ponderon-silverfull  2009

                           
1   RobboLito 0.085g3 x64  +76 Elo  +17/=34/-5 60.71%   34.0/56
2   Rybka 3                -76 Elo  +5/=34/-17 39.29%   22.0/56

Code: Select all

Robbog3x64-R3x64SP-ponderoff-silverfull  2009

                           
1   Rybka 3                -38 Elo  +28/=55/-17 55.50%   55.5/100
2   RobboLito 0.085g3 x64  +38 Elo  +17/=55/-28 44.50%   44.5/100

How can you compare the 2 performances(ponder ON/OFF) if the games played are not the same? 100 games played with ponder OFF while only 56 with ponder ON.

Also again as in your other post your program that produces these outputs is crazy. How can it have Rybka(on the ponder OFF) list to win with 55.5/100 and say -38 Elo??

[George Tsavdaris]

Based on a huge sample (150k+) of positions from top-level correspondence play, I compared Rybka 3 on one core to Robbolito. At one second per position, Robbolito matched the move choices better than R3 by nearly one percentage point, roughly 50 Elo, in line with your results. However at one minute per position (using a sampling of 3k positions), Rybka scored better than Robbolito by an even larger margin.

The margin of error in the first case(150000 positions-short time control) is way smaller than in the second case(3000 positions-long time control). So you can't really compare performances and say that in longer time controls Rybka performs better.

[lkaufman]

After about 10k positions at 1 minute, Rybka still had a small lead over Robbo. The margin of error is hard to calculate because many positions occur more than once, so the move choice is not just right or wrong, but could be for example 60% right. So I would say that I am not sure which program is stronger (based on this test) at one minute, but I am pretty sure that Rybka is relatively stronger at one minute than at one second.