Houdini 3

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Houdini 3

Post by Dr.Wael Deeb »

marijan wrote:
Houdini wrote:
melajara wrote:Hi Robert,

Don't forget to dedicate some resources for testing at long time controls too as some people here are implying Houdini 1.5 is stronger than version 2 at the regular OTB time control (40 moves in 120 minutes).

Best regards
Mel, some of the most vocal of the "some people here" have never actually tested Houdini 2.....

Robert
Well said Robert!

Here is results from my 60 min per engine test...

i3 370 M
2 cores
512 hash
reversible starting pos 8 moves

Image

Image
So according to your testing there is a gap of 93 Elo between Houdini 1.5a & Houdini 2.0c :?: :?: :?: :?:
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3

Post by MM »

lkaufman wrote:
Mike S. wrote:[quote="mclane & Stockfish.

Maybe my explanation is even more simple but true: Rybka, Ippo & Co are simply better. :mrgreen: The weaker engines just benefit from the bigger draw rates at big depths/long time controls.
This might be a factor for Houdini, but certainly Ivanhoe (Ippo), Rybka, and all other Ippo-related programs are not stronger than Komodo and not measurably stronger than SF except at bullet chess, so this cannot be the explanation here.

There seem to be two theories to explain the observed scaling behavior:
1. Komodo (and perhaps also SF) are more intelligent but slower, and that this tradeoff usually (but not always) favors the fast programs in blitz and the smart programs at long time controls.
2. For whatever reason, the search in Komodo (and perhaps SF) scales better than the search in the Rybka/Ippo family.

Both could be true. If the second is true, can anyone suggest WHY SF might scale better in search than Ivanhoe et al?
I don't think ''intelligence'' is the right word. There are two different approaches to the search in my view.

Ippo family engines have a search for which they find tactical moves in a very short time.
I think it is a question of search (and evaluation). Like humans. Some humans search mainly for tactical moves.
I'm not a programmer but i think it depends by the evaluation that an engine gives to each move that it analyses.
In this way it could happen that all the moves that don't give a ''break'' in the evaluation are discarded or analysed less time (and perhaps they are good positional moves).
If this would be true, it is logical that these engines find more often and more quickly tactical moves.

On the other hand Komodo plays mainly positionally.
Komodo often manouevres his pieces and pawns for many moves without playing any tactical move.

Probably it depends by the fact that Komodo gives a special evaluation on some moves that lead to some positional patterns.

I mean, probably during the search, Komodo seems to make the opposite of the engines of the ippo family.

Then Komodo analyzes more the ''positional moves'' (probably because it consider them better than other moves with its evaluation) and gives less time or discards some tactical moves.

In fact, sometimes Komodo overlooks some tactical moves, and it depends by the search or sometimes by the evaluation.

If all i wrote would be true, it is logical that in very fast time control, ippo family engines excel.

With more time to think, a positional engine like Komodo has 2 advantages:

1. What Komodo overlooks or evaluates bad in a few seconds can be seen well with more time available.

2.The weight of positional play of Komodo increases a lot and becomes more important than its (relative) weakness in tactics.

Generally speaking i think that the strenght of Komodo is that it plays nice positional and logical moves. It seems that it uses the search just to verify that everything is ok.

On the other hand the strenght of ippo family engines is the approach of the search (i think built mainly for tactical moves) and the evaluation that allows them to find sometimes tactical moves apparently hidden.

In long time control, especially in very long time control, tactics ability is almost useless because the opponent (if he has a good search and evaluation) has all the time to see it.

So the thing that has more importance is the positional/strategical sensibility.


Best Regards
MM
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3

Post by MM »

Houdini wrote:Hello Bram,

Question 1: Compared to Houdini 1.5a I get the following results:
- 27,000 games against 9 other engines on ultra-fast TC gives +45 Elo
- 3,600 games against 9 other engines on blitz TC gives +46 Elo

Question 2: Nothing has been decided yet.

Greetings,
Robert
Hello Mr Houdart,

After more than 2200 games 20/40 CEGT says that 2.0c has 4 points more than 1.5a (both with 4 cores).

After more than 1200 games 1.5a has 10 points more than 2.0c (both with 1 single core).


http://www.husvankempen.de/nunn/40_40%2 ... liste.html


I remember you claimed many times that 2.0 was about 25 elo stronger than 1.5a.

In my own tests, based on thousand games, the difference was around 5-10 elo in favor of 2.0.

So i wonder: which kind of testing method do you use? Which opening book do you use? Which Hardware do you use?

P.S. i mailed you recently but i didn't get any answer (i am a customer of yours).

If you want you can reply me by mail.

Thank you.

Best Regards
MM
jpqy
Posts: 554
Joined: Thu Apr 24, 2008 9:31 am
Location: Belgium

Re: Houdini 3

Post by jpqy »

To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3

Post by MM »

jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
MM
Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 11:58 am
Location: Antalya/Turkey

Re: Houdini 3

Post by Sedat Canbaz »

MM wrote:
jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
Dear Maurizio,

I don't understand what is your point exactly...

If you really don't see the Houdini Elo difference,then it seems there is something wrong in your test

Maybe i am wrong..so can you inform me please your hardware test conditions ?

Just my 2 cents over this issue,
Honestly i am very happy to be a Houdini customer
And there is no doubt that Houdini is the world's strongest engine

Another interesting note is that,some people debated that Rybka seems to a clone
But then really i wonder:
-if its a clone,then why Rybka even without pawn,its much stronger than almost all original engines ?

Btw,i am happy also that we have Critter,Stockfish,Strelka,Ivanhoe,Komodo,Fire...

Otherwise i am afraid that Houdini's price will go up :)

Greetings,
Sedat
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3

Post by MM »

Sedat Canbaz wrote:
MM wrote:
jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
Dear Maurizio,

I don't understand what is your point exactly...

If you really don't see the real Houdini Elo difference,then it seems there is something wrong in your test

Maybe i am wrong..so can you inform me please your hardware test conditions ?

Just my 2 cents over this issue,
Honestly i am very happy to be a Houdini customer
And there is no doubt that Houdini is the world's strongest engine

Another interesting note is that,some people debated that Rybka seems to a clone
But then really i wonder:
-if its a clone,then why Rybka even without pawn,its much stronger than almost all original engines ?

Btw,i am happy also that we have Critter,Stockfish,Strelka,Ivanhoe,Komodo,Fire...

Otherwise i am afraid that Houdini's price will go up :)

Greetings,
Sedat
Hello Sedat, sorry but i think there is a misunderstanding.

Me i was the one who asked Mr Houdart to specify (please) the conditions under which he tested Houdini 2.0 because he claimed many times that 2.0 is about 25 elo stronger than 1.5a.

CEGT reports different results.

That's why i asked.

I am interested in:

exact time control (s)
number of games
opening book (s)
contempt
tablebases
hardware
number of cores used

Ipon Chess of Ingo Bauer shows a difference (in favor of 2.0) of 7 elo after 4000 games (Under Rating List) so CEGT it's not the only one rating list that sees 2.0 and 1.5a so close.

http://www.inwoba.de/

I don't claim that my results are correct but i see that my results are almost identical to the ones of Ipon Chess.


Best Regards
MM
User avatar
Dr.Wael Deeb
Posts: 9773
Joined: Wed Mar 08, 2006 8:44 pm
Location: Amman,Jordan

Re: Houdini 3

Post by Dr.Wael Deeb »

MM wrote:
Sedat Canbaz wrote:
MM wrote:
jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
Dear Maurizio,

I don't understand what is your point exactly...

If you really don't see the real Houdini Elo difference,then it seems there is something wrong in your test

Maybe i am wrong..so can you inform me please your hardware test conditions ?

Just my 2 cents over this issue,
Honestly i am very happy to be a Houdini customer
And there is no doubt that Houdini is the world's strongest engine

Another interesting note is that,some people debated that Rybka seems to a clone
But then really i wonder:
-if its a clone,then why Rybka even without pawn,its much stronger than almost all original engines ?

Btw,i am happy also that we have Critter,Stockfish,Strelka,Ivanhoe,Komodo,Fire...

Otherwise i am afraid that Houdini's price will go up :)

Greetings,
Sedat
Hello Sedat, sorry but i think there is a misunderstanding.

Me i was the one who asked Mr Houdart to specify (please) the conditions under which he tested Houdini 2.0 because he claimed many times that 2.0 is about 25 elo stronger than 1.5a.

CEGT reports different results.

That's why i asked.

I am interested in:

exact time control (s)
number of games
opening book (s)
contempt
tablebases
hardware
number of cores used

Ipon Chess of Ingo Bauer shows a difference (in favor of 2.0) of 7 elo after 4000 games (Under Rating List) so CEGT it's not the only one rating list that sees 2.0 and 1.5a so close.

http://www.inwoba.de/

I don't claim that my results are correct but i see that my results are almost identical to the ones of Ipon Chess.


Best Regards
As my memory still serves me well yet,I think that Robert commited the same fatal mistkae that The author of Ruffian did in the past....

Ruffian's author released the extremely strong for it's time Ruffian 1.05 and then went commercial....the commercial version was an improvement after all,but not that much....

In Houdini's case,the difference between 1.5a & 2.0c is 10 Elo at best....

So Mr.Houdini should have released Houdini 1.5a as a commercial version and improve furhter the next version....
I give Robert F in computer chess marketing regards,
Dr.D
_No one can hit as hard as life.But it ain’t about how hard you can hit.It’s about how hard you can get hit and keep moving forward.How much you can take and keep moving forward….
Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 11:58 am
Location: Antalya/Turkey

Re: Houdini 3

Post by Sedat Canbaz »

MM wrote:
Sedat Canbaz wrote:
MM wrote:
jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
Dear Maurizio,

I don't understand what is your point exactly...

If you really don't see the real Houdini Elo difference,then it seems there is something wrong in your test

Maybe i am wrong..so can you inform me please your hardware test conditions ?

Just my 2 cents over this issue,
Honestly i am very happy to be a Houdini customer
And there is no doubt that Houdini is the world's strongest engine

Another interesting note is that,some people debated that Rybka seems to a clone
But then really i wonder:
-if its a clone,then why Rybka even without pawn,its much stronger than almost all original engines ?

Btw,i am happy also that we have Critter,Stockfish,Strelka,Ivanhoe,Komodo,Fire...

Otherwise i am afraid that Houdini's price will go up :)

Greetings,
Sedat
Hello Sedat, sorry but i think there is a misunderstanding.

Me i was the one who asked Mr Houdart to specify (please) the conditions under which he tested Houdini 2.0 because he claimed many times that 2.0 is about 25 elo stronger than 1.5a.

CEGT reports different results.

That's why i asked.

I am interested in:

exact time control (s)
number of games
opening book (s)
contempt
tablebases
hardware
number of cores used

Ipon Chess of Ingo Bauer shows a difference (in favor of 2.0) of 7 elo after 4000 games (Under Rating List) so CEGT it's not the only one rating list that sees 2.0 and 1.5a so close.

http://www.inwoba.de/

I don't claim that my results are correct but i see that my results are almost identical to the ones of Ipon Chess.


Best Regards

Hello dear Maurizio,

Thank you for your replay...

Oh...yes,now its more clear

In my opinion,the Elo difference between Houdini 2.0c and Houdini 1.5a is around 20-25 Elo

SWCR (40/10):21 Elo difference

Code: Select all

           NAME / version of engine       ELO    +    -   GAM    SC   OP     DR
  ------------------------------------------------------------------------------
   1     1 Houdini 2.0c x64               3019   18   18  1400   79%  2777   28% 
   -     2 Houdini 1.5 x64                2999   14   14  2320   78%  2772   29%


SCCT Auto232(4m+2s):expecting to be around 20-25 Elo difference too

Code: Select all

Rank  Name                        Elo    +    -   games score oppo. draws
   1 Houdini 2.0c Pro x64 6c      3441   21   21   632   61%  3379   53%
   3 Houdini 2.0b Pro x64 6c      3425   17   17  1049   68%  3312   46%
   8 Houdini 2.0b Pro x64 4c      3355   17   17  1030   51%  3352   47%
  10 Houdini 1.5a x64 4c          3352   17   17  1086   62%  3280   48% 
About IPON 4000 games per player,
It sounds perfect, but can you prove that ?
Plus,i will be very happy,if i will access to IPON's openings and games too (with annotations if possible)

About CEGT,
I have big respect to theirs work,but however i think its quite normal to see a such Elo difference under adapted time controls and using many various openings and different hardwares


Best Wshes,
Sedat
MM
Posts: 766
Joined: Sun Oct 16, 2011 11:25 am

Re: Houdini 3

Post by MM »

Sedat Canbaz wrote:
MM wrote:
Sedat Canbaz wrote:
MM wrote:
jpqy wrote:To have the right strenght difference you have to test the different Split Depth's
I get in my results Houdini 2.0 clearly stronger.
If you look to these lists and they get so small or no difference in ELO means they just use default settings.
Houdini has as default Split Depth 10 ..this is tuned for a system as a Q6600
When you play with SD=10 on a slower system..Houdini will play weaker ,even he stay strongest engine!
Lower the SD and you get better result.
On my i7 970 i have tested from SD=10 to SD=14 and SD=12 is clearly stronger when i compare with default.
When you have find the right Split Depth you will get better results against other engines.
On my little laptop i have to use SD=9 to get best results.
These lists are nice but don't show the real strenght off these engines.
People who say i get only 5Elo difference..well you are not using the best Split Depth for your system! :wink:

JP.
I tested with the correct split depth. I read the manual of Mr Houdart and made the test.

Best Regards
Dear Maurizio,

I don't understand what is your point exactly...

If you really don't see the real Houdini Elo difference,then it seems there is something wrong in your test

Maybe i am wrong..so can you inform me please your hardware test conditions ?

Just my 2 cents over this issue,
Honestly i am very happy to be a Houdini customer
And there is no doubt that Houdini is the world's strongest engine

Another interesting note is that,some people debated that Rybka seems to a clone
But then really i wonder:
-if its a clone,then why Rybka even without pawn,its much stronger than almost all original engines ?

Btw,i am happy also that we have Critter,Stockfish,Strelka,Ivanhoe,Komodo,Fire...

Otherwise i am afraid that Houdini's price will go up :)

Greetings,
Sedat
Hello Sedat, sorry but i think there is a misunderstanding.

Me i was the one who asked Mr Houdart to specify (please) the conditions under which he tested Houdini 2.0 because he claimed many times that 2.0 is about 25 elo stronger than 1.5a.

CEGT reports different results.

That's why i asked.

I am interested in:

exact time control (s)
number of games
opening book (s)
contempt
tablebases
hardware
number of cores used

Ipon Chess of Ingo Bauer shows a difference (in favor of 2.0) of 7 elo after 4000 games (Under Rating List) so CEGT it's not the only one rating list that sees 2.0 and 1.5a so close.

http://www.inwoba.de/

I don't claim that my results are correct but i see that my results are almost identical to the ones of Ipon Chess.


Best Regards

Hello dear Maurizio,

Thank you for your replay...

Oh...yes,now its more clear

In my opinion,the Elo difference between Houdini 2.0c and Houdini 1.5a is around 20-25 Elo

SWCR (40/10):21 Elo difference

Code: Select all

           NAME / version of engine       ELO    +    -   GAM    SC   OP     DR
  ------------------------------------------------------------------------------
   1     1 Houdini 2.0c x64               3019   18   18  1400   79%  2777   28% 
   -     2 Houdini 1.5 x64                2999   14   14  2320   78%  2772   29%


SCCT Auto232(4m+2s):expecting to be around 20-25 Elo difference too

Code: Select all

Rank  Name                        Elo    +    -   games score oppo. draws
   1 Houdini 2.0c Pro x64 6c      3441   21   21   632   61%  3379   53%
   3 Houdini 2.0b Pro x64 6c      3425   17   17  1049   68%  3312   46%
   8 Houdini 2.0b Pro x64 4c      3355   17   17  1030   51%  3352   47%
  10 Houdini 1.5a x64 4c          3352   17   17  1086   62%  3280   48% 
About IPON 4000 games per player,
It sounds perfect, but can you prove that ?
Plus,i will be very happy,if i will access to IPON's openings and games too (with annotations if possible)

About CEGT,
I have big respect to theirs work,but however i think its quite normal to see a such Elo difference under adapted time controls and using many various openings and different hardwares


Best Wshes,
Sedat
Hi Sedat,

to check ipon chess you should go here:

http://www.inwoba.de/index.html

then click on ''rating list'' and scroll down to ''full list''.

As regards the games and other infos that don't appear into the site, you should ask to Mr Bauer.

As regards CEGT, i think that 1200 games are not enough to be sure at 100% about a result but the fact that 1.5a scores +10 elo versus 2.0 at single core impresses me a little and, with ipon chess and my results, this is the reason why i asked Mr Houdart some more informations about his testing conditions.


Best Regards
MM