100 long games Rybka 4 vs Houdini 1.03a

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Milos »

Uri Blass wrote:This is simply not correct.

20 elo is not a year and we usually get more than 20 elo per year in software(if you compare rybka with the best software of 2005 you get clearly more than 100 elo) and Rybka clearly earns more speed from 64 bits relative to IvanHoe.
It is exactly correct today and for Rybka (talking about 2005 in 2010 is just wrong). Rybka improved less than 40 elo for more than 2 years.
Anyone thinking Vas could improve more than 40 elo if wanted to is nothing but a dreamer or a fan.
The reason is that Vasik did not care to optimize rybka for 32 bits so the gap between 32 bits and 64 bits is bigger for rybka.

I believe that
Vas can make Rybka 32 bits 20-30% faster but he does not care about it because most customers who use 32 bits do not care about rating.
You believe it, but that's a classic example of wishful thinking. Everyone assumes they know for sure what Vas is capable of, and like always reality strongly refutes them...
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Dr.Wael Deeb »

Milos wrote:
Uri Blass wrote:This is simply not correct.

20 elo is not a year and we usually get more than 20 elo per year in software(if you compare rybka with the best software of 2005 you get clearly more than 100 elo) and Rybka clearly earns more speed from 64 bits relative to IvanHoe.
It is exactly correct today and for Rybka (talking about 2005 in 2010 is just wrong). Rybka improved less than 40 elo for more than 2 years.
Anyone thinking Vas could improve more than 40 elo if wanted to is nothing but a dreamer or a fan.
The reason is that Vasik did not care to optimize rybka for 32 bits so the gap between 32 bits and 64 bits is bigger for rybka.

I believe that
Vas can make Rybka 32 bits 20-30% faster but he does not care about it because most customers who use 32 bits do not care about rating.
You believe it, but that's a classic example of wishful thinking. Everyone assumes they know for sure what Vas is capable of, and like always reality strongly refutes them...
I'm afraid that I have to agree with Milos here Uri....
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
User avatar
Graham Banks
Posts: 41461
Joined: Sun Feb 26, 2006 10:52 am
Location: Auckland, NZ

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Graham Banks »

Uri Blass wrote:I believe that Vas can make Rybka 32 bits 20-30% faster but he does not care about it because most customers who use 32 bits do not care about rating.
Why shouldn't 32-bit users care about the strength relative to other engines? I disagree with that part of your statement.

Cheers,
Graham.
gbanksnz at gmail.com
Gino Figlio
Posts: 95
Joined: Sun Jan 10, 2010 6:10 am
Location: Lamar, Colorado, USA

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Gino Figlio »

Laskos wrote:
...All in all, this IvanHoe is ~20 +/-5 Elo points stronger than Rybka 4 with 95% confidence, it has a "Game" mode and an "Analysis" mode, uses endgame bitbases, and is clearly ahead of Rybka 4 by at least a year if we are to judge the progress of Rybka. It is much more useful in Game mode (Playchess for example) and Analysis mode, because it doesn't have infinite time analysis bug as Rybka 4 has (stuck at some depth).

Do you want more? I can continue the test to bring error margins even smaller. Nowhere here such tests were conducted, and probably some folks here (paid or not) will flock to argue that Rybka is better at some outlandish time controls, with all its bugs!

For everyone here, latest free Ippo* engines are the best engines out there for game and analysis, at least a year ahead of the best commercial engines. Period.

Kai
I fully agree about Rybka not being the best anymore.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Laskos »

Gino Figlio wrote:
Laskos wrote:
...All in all, this IvanHoe is ~20 +/-5 Elo points stronger than Rybka 4 with 95% confidence, it has a "Game" mode and an "Analysis" mode, uses endgame bitbases, and is clearly ahead of Rybka 4 by at least a year if we are to judge the progress of Rybka. It is much more useful in Game mode (Playchess for example) and Analysis mode, because it doesn't have infinite time analysis bug as Rybka 4 has (stuck at some depth).

Do you want more? I can continue the test to bring error margins even smaller. Nowhere here such tests were conducted, and probably some folks here (paid or not) will flock to argue that Rybka is better at some outlandish time controls, with all its bugs!

For everyone here, latest free Ippo* engines are the best engines out there for game and analysis, at least a year ahead of the best commercial engines. Period.

Kai
I fully agree about Rybka not being the best anymore.
Thanks, I am a little disappointed by Uri, he is usually correct and factual, but came here with words and suppositions. Thanks to Graham too for objecting.

Meanwhile I continued the test (by the way, there are better testers than me with 64 bit OS and heavy hardware having the same results). I reduced the error margins to 4 Elo points 95% confidence, 2 Elo points 68% confidence.


Code: Select all

    Program                            Score        %    Av.Op.  Elo    +   -    Draws

  1 IvanHoe B50hCx32               : 9168.5/17300  53.0   3190   3210    4   4   30.4 %
  2 Rybka 4w32                     : 8131.5/17300  47.0   3210   3190    4   4   30.4 %

21 +/- 4
Elo points advantage for IvanHoe without endgame bitbases 95% confidence, 2 Elo points error 68% confidence. With bitbases it is 25 +/- 4 Elo points advantage 95% confidence. LOS is so close to 100% for IvanHoe that it's too long to post it here in a line of 9 and a dot :).
Albert Silver
Posts: 3019
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Albert Silver »

Laskos wrote:
Gino Figlio wrote:
Laskos wrote:
...All in all, this IvanHoe is ~20 +/-5 Elo points stronger than Rybka 4 with 95% confidence, it has a "Game" mode and an "Analysis" mode, uses endgame bitbases, and is clearly ahead of Rybka 4 by at least a year if we are to judge the progress of Rybka. It is much more useful in Game mode (Playchess for example) and Analysis mode, because it doesn't have infinite time analysis bug as Rybka 4 has (stuck at some depth).

Do you want more? I can continue the test to bring error margins even smaller. Nowhere here such tests were conducted, and probably some folks here (paid or not) will flock to argue that Rybka is better at some outlandish time controls, with all its bugs!

For everyone here, latest free Ippo* engines are the best engines out there for game and analysis, at least a year ahead of the best commercial engines. Period.

Kai
I fully agree about Rybka not being the best anymore.
Thanks, I am a little disappointed by Uri, he is usually correct and factual, but came here with words and suppositions. Thanks to Graham too for objecting.

Meanwhile I continued the test (by the way, there are better testers than me with 64 bit OS and heavy hardware having the same results). I reduced the error margins to 4 Elo points 95% confidence, 2 Elo points 68% confidence.


Code: Select all

    Program                            Score        %    Av.Op.  Elo    +   -    Draws

  1 IvanHoe B50hCx32               : 9168.5/17300  53.0   3190   3210    4   4   30.4 %
  2 Rybka 4w32                     : 8131.5/17300  47.0   3210   3190    4   4   30.4 %

21 +/- 4
Elo points advantage for IvanHoe without endgame bitbases 95% confidence, 2 Elo points error 68% confidence. With bitbases it is 25 +/- 4 Elo points advantage 95% confidence. LOS is so close to 100% for IvanHoe that it's too long to post it here in a line of 9 and a dot :).
You're missing his point. Rybka 4 gains more out of 64-bit, since that is what it is optimized for, thus if you really want to test the best engine, you should give it the proper conditions.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Laskos »

Albert Silver wrote:
You're missing his point. Rybka 4 gains more out of 64-bit, since that is what it is optimized for, thus if you really want to test the best engine, you should give it the proper conditions.
No, I am not missing anything. All Ippo* engines are bitboard based engines. It's only a supposition to say that they scale differently from 32 to 64 bits. I saw exactly the same scaling as Rybka 4 in the tests of other folks not present in CCC. Fine now?

Kai
Uri Blass
Posts: 10309
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Uri Blass »

Laskos wrote:
Albert Silver wrote:
You're missing his point. Rybka 4 gains more out of 64-bit, since that is what it is optimized for, thus if you really want to test the best engine, you should give it the proper conditions.
No, I am not missing anything. All Ippo* engines are bitboard based engines. It's only a supposition to say that they scale differently from 32 to 64 bits. I saw exactly the same scaling as Rybka 4 in the tests of other folks not present in CCC. Fine now?

Kai
I read that rybka4 earns more speed from 64 bits relative to Ippo* engines and I think that people can test it.

I did not buy rybka4 so I cannot test it but I trust the people who claimed it.

I also read in the programming subforum that there are special tricks
to make stockfish(that is also a bitboard engine) faster for 32 bits and I read that the gap for stockfish between 32 bits and 64 bits is smaller than the gap for rybka.

I think that it seems clear that Vas can also do rybka faster for 32 bits but he does not care about it.

I thought that a possible explanation is that the majority of the 32 bit users do not care
Maybe this explanation is simply wrong and Vas simply does not care about the customers but
it does not change the fact that the fair way to compare bitboard engines is when both use 64 bits and not 32 bits(because part of the programmers of 64 bits engines are not interested in optimizations for 32 bits)
Uri Blass
Posts: 10309
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Uri Blass »

Milos wrote:
Uri Blass wrote:This is simply not correct.

20 elo is not a year and we usually get more than 20 elo per year in software(if you compare rybka with the best software of 2005 you get clearly more than 100 elo) and Rybka clearly earns more speed from 64 bits relative to IvanHoe.
It is exactly correct today and for Rybka (talking about 2005 in 2010 is just wrong). Rybka improved less than 40 elo for more than 2 years.
Anyone thinking Vas could improve more than 40 elo if wanted to is nothing but a dreamer or a fan.
The reason is that Vasik did not care to optimize rybka for 32 bits so the gap between 32 bits and 64 bits is bigger for rybka.

I believe that
Vas can make Rybka 32 bits 20-30% faster but he does not care about it because most customers who use 32 bits do not care about rating.
You believe it, but that's a classic example of wishful thinking. Everyone assumes they know for sure what Vas is capable of, and like always reality strongly refutes them...
I do not know how much rybka improved and we only can compare
the public version.

reality does not prove nothing because
we do not know if Vas release the strongest rybka that he has.

I expect more than 20 elo improvement in the next year and I do not talk about improvement in public rybka but about improvement in top engines like stockfish.

I do not like Ippolit derivatives because I prefer engine with known authors.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: 100 long games Rybka 4 vs Houdini 1.03a

Post by Laskos »

Uri Blass wrote:
Laskos wrote:
Albert Silver wrote:
You're missing his point. Rybka 4 gains more out of 64-bit, since that is what it is optimized for, thus if you really want to test the best engine, you should give it the proper conditions.
No, I am not missing anything. All Ippo* engines are bitboard based engines. It's only a supposition to say that they scale differently from 32 to 64 bits. I saw exactly the same scaling as Rybka 4 in the tests of other folks not present in CCC. Fine now?

Kai
I read that rybka4 earns more speed from 64 bits relative to Ippo* engines and I think that people can test it.

I did not buy rybka4 so I cannot test it but I trust the people who claimed it.

I also read in the programming subforum that there are special tricks
to make stockfish(that is also a bitboard engine) faster for 32 bits and I read that the gap for stockfish between 32 bits and 64 bits is smaller than the gap for rybka.

I think that it seems clear that Vas can also do rybka faster for 32 bits but he does not care about it.

I thought that a possible explanation is that the majority of the 32 bit users do not care
Maybe this explanation is simply wrong and Vas simply does not care about the customers but
it does not change the fact that the fair way to compare bitboard engines is when both use 64 bits and not 32 bits(because part of the programmers of 64 bits engines are not interested in optimizations for 32 bits)
Uri, most serious (many games) tests I saw (not in CCC) show that Ippo* engines scale very similarly to Rybka 4 from 32 to 64 bits. Maybe 4-5 Elo points one way or another. I do not think that Vas was not caring about the 32 bit engine.

I have a 32 bit reliable OS and 2 threads, that is all I can do for now.

Why do you make such unsupported statements? I often appreciate your posts for being factual and intuitive, but now the facts are on my side.

Kai