Testing Houdini 3.0 Pro

[Tomcass]

TESTING Houdini 3 Pro = 60 minutes/game. THIRD LEG

I7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Rybka 3.0
Time control: 60 minutes/game
Ponder: Off
No tablebases

50 BNG (Billion Nodes per Game for Houdini).

2010-01, Blitz 60m 0

1 Ivanhoe B50kBx64p(x4)BO +2/=17/-1 52.50% 10.5/20
2 Houdini 3 Pro x64_4 +1/=17/-2 47.50% 9.5/20

http://www.mediafire.com/?75zi11bqpz3givg

... and this is a big surprise for me!. The old PeterPan's Ivanhoe compile has been able to beat the solid Number One Houdini 3.0 Pro. I know that the number of games is very small and that the book selected Rybka 3.0 offers some advantage for Ivan, but most of the games have been extremely tough. Only one defeat after 20 games against this monster is a great result for Ivan.

I will repeat this test (when the heat of my computer goes down) only to try to avoid -or minimize- statistical noise, but I feel impressed by the performance of this Ivanhoe compile.

Best regards from Barcelona.

Tom.

Thanks Tom

One of the compiles of Ivanhoe who ia a permanent participant in my rating list.....

However,there is another extremely strong compile by PeterPan that can even perform better.........
Dr.D

Hi Dr. Deeb,

I can guess what you mean ... but please avoid me the risk to fail. What is this extremely strong PeterPan compile?.

Best regards,

Tom.

Hi Tom

Definitely Ivanhoe-B PP-0.1 x64 .....

Extremely strong compile and I test at relatively long time controls:
20 minutes + 20 seconds increment....
Best regards,
Dr.D

Thanks for the tip Dr. Deeb!. I will re-test this compile.

Tom.

[Dr.Wael Deeb]

TESTING Houdini 3 Pro = 60 minutes/game. THIRD LEG

I7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Rybka 3.0
Time control: 60 minutes/game
Ponder: Off
No tablebases

50 BNG (Billion Nodes per Game for Houdini).

2010-01, Blitz 60m 0

1 Ivanhoe B50kBx64p(x4)BO +2/=17/-1 52.50% 10.5/20
2 Houdini 3 Pro x64_4 +1/=17/-2 47.50% 9.5/20

http://www.mediafire.com/?75zi11bqpz3givg

... and this is a big surprise for me!. The old PeterPan's Ivanhoe compile has been able to beat the solid Number One Houdini 3.0 Pro. I know that the number of games is very small and that the book selected Rybka 3.0 offers some advantage for Ivan, but most of the games have been extremely tough. Only one defeat after 20 games against this monster is a great result for Ivan.

I will repeat this test (when the heat of my computer goes down) only to try to avoid -or minimize- statistical noise, but I feel impressed by the performance of this Ivanhoe compile.

Best regards from Barcelona.

Tom.

Thanks Tom

One of the compiles of Ivanhoe who ia a permanent participant in my rating list.....

However,there is another extremely strong compile by PeterPan that can even perform better.........
Dr.D

Hi Dr. Deeb,

I can guess what you mean ... but please avoid me the risk to fail. What is this extremely strong PeterPan compile?.

Best regards,

Tom.

Hi Tom

Definitely Ivanhoe-B PP-0.1 x64 .....

Extremely strong compile and I test at relatively long time controls:
20 minutes + 20 seconds increment....
Best regards,
Dr.D

Thanks for the tip Dr. Deeb!. I will re-test this compile.

Tom.

Looking forward to your results Tom.....
Dr.D

[beram]

Anyway, this is the first time since Houdini 3.0 appeared that after 20 games in a row, at any time control, Houdini 3.0 has been able to beat ONLY ONCE to his rival. At least AFAIK.

Actually it has happened many times in my own tests. An example? Tie together all the Noomen tests, so you have 90 positions; when Houdini 3 comes about the 61 one (122 game), it cannot usually win for about 25 games after against Komodo 5 (or either Critter), to then win 15 in a row just after.

Naturally it is not always repetible, because randomness is a factor, but there are some positions where Houdini has more trouble than others, as every other engine.

Thanks for this info, Fabio. In all the tests I followed I have not been able to find such circumstance for Houdini 3.0. I'll keep testing.

Tom.

I have found something similar
When I test Houdini 3 against Komodo 3 with Noomen testsuite 2012, Houdini scores 60-62% on all different TC's. But when I use my own testset Houdini than scores 68-70%
That doesnt necessarily mean that Houdini is always better with that positions, because when I test with Critter 1.6 on different TC's the Houdini 3 result for both testsuites results are equally the same around 60%

grts Bram

[Tomcass]

TESTING Houdini 3 Pro = 60 minutes/game. THIRD LEG/b

After 40 games ...

I7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Rybka 3.0
Time control: 60 minutes/game
Ponder: Off
No tablebases

50 BNG (Billion Nodes per Game for Houdini).

2010-01, Blitz 60m 0

1 Ivanhoe B50kBx64p(x4)BO +5/=31/-4 51.25% 20.5/40
2 Houdini 3 Pro x64_4 +4/=31/-5 48.75% 19.5/40

Games 21 to 40: http://www.mediafire.com/?7fyy9yj6i6a6f86

... Ivanhoe B50kB still ahead of Houdini 3.0 Pro

I will keep testing until there is no statistical doubt about this test. Or until my computer crashes!.

Best regards from Barcelona.

Tom.

[Carlos777]

TESTING Houdini 3 Pro = 60 minutes/game. THIRD LEG/b

After 40 games ...

I7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Rybka 3.0
Time control: 60 minutes/game
Ponder: Off
No tablebases

50 BNG (Billion Nodes per Game for Houdini).

2010-01, Blitz 60m 0

1 Ivanhoe B50kBx64p(x4)BO +5/=31/-4 51.25% 20.5/40
2 Houdini 3 Pro x64_4 +4/=31/-5 48.75% 19.5/40

Games 21 to 40: http://www.mediafire.com/?7fyy9yj6i6a6f86

... Ivanhoe B50kB still ahead of Houdini 3.0 Pro

I will keep testing until there is no statistical doubt about this test. Or until my computer crashes!.

Best regards from Barcelona.

Tom.

Very impressive. We can't draw to any conclusion from this match so far statistically speaking, but still impressive nonetheless.

Thanks for the test Tom. This takes a lot of time and patience by your side. I hope your PC stays well after so intensive testing.

Greetings,
Carlos

[Lavir]

I7 975 3.33 Ghz.
4 real cores
GUI: Fritz 12
Book: Rybka 3.0
Time control: 60 minutes/game
Ponder: Off
No tablebases

There's a reason why you use a so long book as Rybka 3?

With so long lines it's obvious the weaker engine gets a lot of "help" because the chances for it to go wrong in the earlier phases of the game are circumnvented by the book lines and the chances of the stronger engine are in the same way (and viceversa) reduced; usually, also in tournaments with a lot of emphasis on book play, the side with the stronger engine will try to put the weaker engine side out of book (even with a slight inferior position) just for this, while the weaker engine side will try to do just the opposite. It's obvious that the less the chances to go wrong really helps the engine that is (at last "on paper") inferior.

This is well seen in the amount of draws in the test, atm. With books with so long lines as the Rybka 3 one, it is very difficult to ascertain pure engines strength, even less without reversed line play.

If your intention is to test the real strength difference between H3 and Ivanhoe at long time control (that's somewhat even worse just for the book used), a so long book it is a bad idea, IMO.

[Houdini]

This is well seen in the amount of draws in the test, atm.

Exact, if Houdini 3 plays 31 draws out of 40 games against an opponent about 100 to 130 Elo weaker, you know for sure that the playing conditions cannot be standard.
That will obviously not prevent people to refer this Ivanhoe B50kBx64p compile as the version "that was nearly as good as Houdini 3" for years to come.

Robert

[Uri Blass]

This is well seen in the amount of draws in the test, atm.

Exact, if Houdini 3 plays 31 draws out of 40 games against an opponent about 100 to 130 Elo weaker, you know for sure that the playing conditions cannot be standard.
That will obviously not prevent people to refer this Ivanhoe B50kBx64p compile as the version "that was nearly as good as Houdini 3" for years to come.

Robert

An interesting question is what is the rating difference if both versions use the best books to help them(of course in this case houdini3 is not going to use Rybka3's book and is going to prefer a small book to get the opponent out of book).

[Dr.Wael Deeb]

This is well seen in the amount of draws in the test, atm.

Exact, if Houdini 3 plays 31 draws out of 40 games against an opponent about 100 to 130 Elo weaker, you know for sure that the playing conditions cannot be standard.
That will obviously not prevent people to refer this Ivanhoe B50kBx64p compile as the version "that was nearly as good as Houdini 3" for years to come
.

Robert

Come on Robert,

Why are you mocking Houdini's Grandfather,ah

After all,they are one big happy family,aren't they

[beram]

This is well seen in the amount of draws in the test, atm.

Exact, if Houdini 3 plays 31 draws out of 40 games against an opponent about 100 to 130 Elo weaker, you know for sure that the playing conditions cannot be standard.
That will obviously not prevent people to refer this Ivanhoe B50kBx64p compile as the version "that was nearly as good as Houdini 3" for years to come.

Robert

An interesting question is what is the rating difference if both versions use the best books to help them(of course in this case houdini3 is not going to use Rybka3's book and is going to prefer a small book to get the opponent out of book).

If both use the best books suited for their style, you can expect the same results as with a neutral testset where each play same position both with white and black.
So you can expect the same ELO difference