Deep Rybka 4.1 vs Houdini 3

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
MM
Posts: 766
Joined: Sun Oct 16, 2011 9:25 am

Deep Rybka 4.1 vs Houdini 3

Post by MM » Mon Oct 22, 2012 9:41 am

Hi all,


http://www.husvankempen.de/nunn/40_40%2 ... ns/10.html.

It is strange, how can Houdini 1.5a 4 cpu perform so much better than Houdini 3 4cpu against Rybka 4.1 4cpu?

I see not so many games played but considering the big difference in strenght between Houdini 3 and Houdini 1.5a it is very surprising to me.

Thank you

Best Regards
MM

S.Taylor
Posts: 8499
Joined: Thu Mar 09, 2006 2:25 am
Location: Jerusalem Israel

Re: Deep Rybka 4.1 vs Houdini 3

Post by S.Taylor » Mon Oct 22, 2012 9:48 am

MM wrote:Hi all,


http://www.husvankempen.de/nunn/40_40%2 ... ns/10.html.

It is strange, how can Houdini 1.5a 4 cpu perform so much better than Houdini 3 4cpu against Rybka 4.1 4cpu?

I see not so many games played but considering the big difference in strenght between Houdini 3 and Houdini 1.5a it is very surprising to me.

Thank you

Best Regards

Are you sure you are reading it correctly and that this is the case?

If so, I would be dissappointed.

MM
Posts: 766
Joined: Sun Oct 16, 2011 9:25 am

Re: Deep Rybka 4.1 vs Houdini 3

Post by MM » Mon Oct 22, 2012 10:09 am

S.Taylor wrote:
MM wrote:Hi all,


http://www.husvankempen.de/nunn/40_40%2 ... ns/10.html.

It is strange, how can Houdini 1.5a 4 cpu perform so much better than Houdini 3 4cpu against Rybka 4.1 4cpu?

I see not so many games played but considering the big difference in strenght between Houdini 3 and Houdini 1.5a it is very surprising to me.

Thank you

Best Regards

Are you sure you are reading it correctly and that this is the case?

If so, I would be dissappointed.
Hi,

http://www.husvankempen.de/nunn/40_40%2 ... ns/10.html


I see that at 40/20 the results are following:

Deep Rybka 4.1 x 64 4 cpu - Houdini 3 x 64 4cpu +5 =34 -11 44%

Deep Rybka 4.1 x 64 4 cpu - Houdini 1.5a x 64 4 cpu +20 =36 -44 38%

If i consider the difference in strenght between Houdini 3 and Houdini 1.5a it seems very strange that 1.5a performs better, although not many games have been played.


Best Regards
MM

User avatar
Houdini
Posts: 1471
Joined: Mon Mar 15, 2010 11:00 pm
Contact:

Re: Deep Rybka 4.1 vs Houdini 3

Post by Houdini » Mon Oct 22, 2012 10:13 am

MM wrote:It is strange, how can Houdini 1.5a 4 cpu perform so much better than Houdini 3 4cpu against Rybka 4.1 4cpu?

I see not so many games played but considering the big difference in strenght between Houdini 3 and Houdini 1.5a it is very surprising to me.
Nothing out of the ordinary.
Rybka 4 is rated 3048, has a performance of 3110 against Houdini 3 (50 games) and of 2997 against Houdini 1.5a (100 games).
With 50 games the error margin is about ±70 Elo, with 100 games it's about ±50 Elo.

Robert

MM
Posts: 766
Joined: Sun Oct 16, 2011 9:25 am

Re: Deep Rybka 4.1 vs Houdini 3

Post by MM » Mon Oct 22, 2012 10:17 am

Houdini wrote:
MM wrote:It is strange, how can Houdini 1.5a 4 cpu perform so much better than Houdini 3 4cpu against Rybka 4.1 4cpu?

I see not so many games played but considering the big difference in strenght between Houdini 3 and Houdini 1.5a it is very surprising to me.
Nothing out of the ordinary.
Rybka 4 is rated 3048, has a performance of 3110 against Houdini 3 (50 games) and of 2997 against Houdini 1.5a (100 games).
With 50 games the error margin is about ±70 Elo, with 100 games it's about ±50 Elo.

Robert
Hi Robert,

do you mean that you expect Houdini 3 4cpu to perform better than Houdini 1.5a 4cpu against Deep Rybka 4.1 4 cpu at 40/20 or generally at long time control?

Thank you

Best Regards
MM

S.Taylor
Posts: 8499
Joined: Thu Mar 09, 2006 2:25 am
Location: Jerusalem Israel

Re: Deep Rybka 4.1 vs Houdini 3

Post by S.Taylor » Mon Oct 22, 2012 10:22 am

If so, that's TERRIBLE. So what have we achieved by waiting a year?

Perhaps Houdini 3 makes up for it by beating others better, but vs Rybka 4.1, that sampler of games shows it to be going down against Rybka straight on.
Is there nothing wrong with the test or results? (besides it not being many games yet).

User avatar
Houdini
Posts: 1471
Joined: Mon Mar 15, 2010 11:00 pm
Contact:

Re: Deep Rybka 4.1 vs Houdini 3

Post by Houdini » Mon Oct 22, 2012 10:28 am

MM wrote:Hi Robert,

do you mean that you expect Houdini 3 4cpu to perform better than Houdini 1.5a 4cpu against Deep Rybka 4.1 4 cpu at 40/20 or generally at long time control?

Thank you

Best Regards
Maurizio,

Houdini 3 is generally about 70 to 75 Elo stronger than Houdini 1.5.
It's not a good idea to get excited about individual match results in rating lists. If Houdini 3 scored relatively poorly against Rybka 4, it has been compensated in other matches.
The most reliable result is the final rating: in the CEGT 40/20 rating list Houdini 3 4CPU is at 3151, Houdini 1.5a 4CPU is at 3081, Deep Rybka 4CPU at 3048.

Robert

User avatar
Houdini
Posts: 1471
Joined: Mon Mar 15, 2010 11:00 pm
Contact:

Re: Deep Rybka 4.1 vs Houdini 3

Post by Houdini » Mon Oct 22, 2012 10:31 am

S.Taylor wrote:If so, that's TERRIBLE. So what have we achieved by waiting a year?

Perhaps Houdini 3 makes up for it by beating others better, but vs Rybka 4.1, that sampler of games shows it to be going down against Rybka straight on.
Is there nothing wrong with the test or results? (besides it not being many games yet).
As said above, in the CEGT 40/20 rating list Houdini 3 4CPU is at 3151, Houdini 1.5a 4CPU is at 3081, Deep Rybka 4CPU at 3048.
That is TERRIBLE indeed... for the other engines :lol:.

Robert

MM
Posts: 766
Joined: Sun Oct 16, 2011 9:25 am

Re: Deep Rybka 4.1 vs Houdini 3

Post by MM » Mon Oct 22, 2012 10:33 am

I think we may need many games to have a clear result but i think that Rybka 4.1 is still, perhaps except Komodo, the best engine for ''understanding'' positions.

So i can understand that Houdini generally can have some trouble facing Rybka at long time control, especially with many cores.

What would surprise me is if Houdini 3 should show a worse result compared to Houdini 1.5a playing Rybka 4.1.

I agree that the strenght of an engine must be calculated on many individual matches but i wouldn't understand if the improvement of Houdini 3 caused (if) a drop in the performance against Rybka 4.1.



Best Regards
MM

User avatar
Houdini
Posts: 1471
Joined: Mon Mar 15, 2010 11:00 pm
Contact:

Re: Deep Rybka 4.1 vs Houdini 3

Post by Houdini » Mon Oct 22, 2012 10:47 am

MM wrote:I think we may need many games to have a clear result but i think that Rybka 4.1 is still, perhaps except Komodo, the best engine for ''understanding'' positions.

So i can understand that Houdini generally can have some trouble facing Rybka at long time control, especially with many cores.

What would surprise me is if Houdini 3 should show a worse result compared to Houdini 1.5a playing Rybka 4.1.

I agree that the strenght of an engine must be calculated on many individual matches but i wouldn't understand if the improvement of Houdini 3 caused (if) a drop in the performance against Rybka 4.1.



Best Regards
In a rating list gauntlet inevitably some individual matches go well, other go poorly.

Against "Deep Rybka 4.1 x64 4CPU" Houdini 3 had a poor run and performed 50 Elo below its rating.
But against "Deep Rybka 4 Exp.61 x64 1CPU" Houdini 3 had a good run and performed 45 Elo above its rating.

It is not smart to attach a lot of importance to these individual results. There is a reason why 1000 games are played for a rating list, not 50.

Robert

Post Reply