Difference between latests Houdini and Komodo engines

[beram]

Dear CCC followers,

I have done some research in the most common and recognised ratinglists and concentrated on the differences between the two latest engines of Houdini and Komodo. The combined outcome you see under here.

overall difference between Houdini 2.0 and 1.5a = +7
overall difference between Komodo 4 en 3 = +16
overall difference between Houdini 2 and Komodo4 = +45

Code: Select all

		H2.0  H1.5 DIF	KOM4	KOM3	DIF	H2-Kom4
CCRL 4/40	xxxx	3267	18	3222	3199	23	45
CCRL 40/40	3243	3256	-13	3207	3216	-9	36
CEGT 4/40	3236	3220	16	3163	3154	9	73
CEGT 20/40	3185	3200	-15	3171	3140	31	14
IPON(5m/3)	3016	3009	7	2975	2965	10	41
SWCR10/40	3017	2998	19	2965	2960	5	52
KECK(5m/3)	3024	3006	18	2961	2946	15	63
SEDA(4m2)	3260	xxxx	11	3208	3192	16	54
NEBB (4m2)	3104	3100	4	3074	3031	43	30

Some comments about extrapolated numbers:
1) CCRL 4/40 hasn’t tested Houdini2.0 at 1core so I have entered the difference between the 4core Houdini engines
2) Sedat hasn’t tested Houdini1.5a at 1 core so I entered here the difference between the 4 core engines who have been tested
3) Sedat tested Houdini 2.0b and 2.0c at 1 core and as both are the same engines I combined the two results into one
4) Sedat hasn’t tested Komodo 4 . Here I gave Komodo4 an extrapolated result of +16 above the tested Komodo 3

Kind regards Bram

[Frank Quisinsky]

Hi Bram,

all the time CCRL have the biggest differences to the others. Nothing against CCRL, CCRL have the highest time control.

It's a bit tricky to compare it:

1. ELOstat / Bayesian
2. Ponder = on / off
3. Different opening systems
4. Different time controls
and many other things ...

40 in 10 in SWCR on clearly faster hardware with ponder = on, is the same as 40 in 20 in CEGT.

But I think we can see a lot in comparing of all this results.

Example:
Now Komodo isn't better on longer time controls, it seems after the latest changes Komodo is better in shorter time controls. And this is very interesting because the Komodo team now all the latest changes and have a good information for further improvements.

And to Houdini:
I used Houdini 1.5 second version, not Houdini 1.5a !!
With Houdini 1.5a I have a lot of problems with large table. Possible that Houdini 1.5a is a bit stronger 1.5 I used. Same for Houdini 2. I used 2.0c all others 2.0 or 2.0b I believe.

Means, not esay to compare all this exactly!

In SWCR I play without large table and my Q9550 processors don't support SSE. So all engines without large tables and without SSE.

Again to CCRL:
I don't saw this ...
Komodo 4 with 40 in 40 ... is around 40 in 16 in comparing with SWCR ... = -9 to Komodo 3? Could be possible, CCRL have it, so it's ok.

Again, after all available results ... now Komodo is stronger in blitz and not with longer time controls.

Thanks for your work and have a nice evening!

Best
Frank

[Sven]

Code: Select all

                H2.0    H1.5    DIF     KOM4    KOM3    DIF     H2-Kom4
CCRL   4/40     xxxx    3267     18     3222    3199     23     45
CCRL  40/40     3243    3256    -13     3207    3216     -9     36
CEGT   4/40     3236    3220     16     3163    3154      9     73
CEGT  20/40     3185    3200    -15     3171    3140     31     14
IPON (5m/3)     3016    3009      7     2975    2965     10     41
SWCR  10/40     3017    2998     19     2965    2960      5     52
KECK (5m/3)     3024    3006     18     2961    2946     15     63
SEDA (4m2)      3260    xxxx     11     3208    3192     16     54
NEBB (4m2)      3104    3100      4     3074    3031     43     30

Just reformatted (better not use TABs).
Sven

[lkaufman]

Just to clarify, I think you are saying that Komodo 4 gained more in blitz than in slow chess compared to Komodo 3. This could very well be true. However compared to Houdini, either Komodo is MUCH closer in slow chess than in blitz, just look at the CEGT data!

[tomgdrums]

Just to clarify, I think you are saying that Komodo 4 gained more in blitz than in slow chess compared to Komodo 3. This could very well be true. However compared to Houdini, either Komodo is MUCH closer in slow chess than in blitz, just look at the CEGT data!

Hi Larry,

I have just purchased Komodo 4 and am enjoying it for analysis purposes!

My other main engine for post game analysis is Houdini 2 (don't worry I always go through my games on my own first before letting the engines loose )

When using Komodo 4 and Houdini in tandem what would you say their respective strengths and weaknesses are? Or to put it another way, where do they fill in each other's gaps?

[beram]

I am just showing data. But when you would interpretate them by looking at the difference between the two Komodos at slow chess level you and I can both see that CEGT is the only one that finds a big improvement. Both SWCR 10/40and CCRL 40/40 show marginal improvements (:-) of CCRL-9 and SWCR+8

I had expected a little more after all the buzzing in october and november

I hope you will improve Komodo more later on
Grts Bram

[ThatsIt]

I've made a comparison between our 3 lists (TOP-9 + reference of the 40/120 only):
http://computerschach.forumfrei.net/t22 ... und-40-120

Code: Select all

                            1      2      3          difference
Engine                     40/4  40/20  40/120  1-->2   1-->3   2-->3 
Houdini 1.5 x64            3225   3200   3138    -25     -87     -62
Rybka 4.0 x64              3172   3128   3125    -44     -47     - 3
Komodo 2.03 x64            3137   3112   3095    -25     -42     -17
Stockfish 2.0 x64          3128   3116   3086    -12     -42     -30
Critter 0.90 x64           3089   3073   3050    -16     -39     -23
Naum 4.2 x64               3030   3024   3016    - 6     -14     - 8
Deep Sjeng ct 2010         2982   2975   2982    - 7     +-0     + 7
Spike 1.4                  2979   2969   2982    -10     + 3     +13
Deep Shredder 12 x64       3003   2983   2981    -20     -22     - 2
...
Fruit 2.2.1                2766   2773   2770 (reference)

Wolfgang just started the Komodo 4 x64 tests for the 40/120 list:
http://cegt.siteboard.eu/f4t189-40-120- ... 0-x64.html

I'm curious about the results.

Best wishes,
G.S.

[lkaufman]

Just to clarify, I think you are saying that Komodo 4 gained more in blitz than in slow chess compared to Komodo 3. This could very well be true. However compared to Houdini, either Komodo is MUCH closer in slow chess than in blitz, just look at the CEGT data!

Hi Larry,

I have just purchased Komodo 4 and am enjoying it for analysis purposes!

My other main engine for post game analysis is Houdini 2 (don't worry I always go through my games on my own first before letting the engines loose )

When using Komodo 4 and Houdini in tandem what would you say their respective strengths and weaknesses are? Or to put it another way, where do they fill in each other's gaps?

In general my opinion is that Houdini will be more likely to be correct when there are hidden tactics in the position, but Komodo will be more likely to be correct when it's purely a question of evaluating positions. Houdini outrates Komodo on the lists primarily because Komodo spends more time in evaluation, and speed is usually decisive in engine vs engine play. This correlates with the notion that Houdini's opinion of the best move will more likely be correct after a few seconds but after some unknown amount of time Komodo should be more likely to be correct. The two programs are rather different so it's certainly good to see both opinions. When I wrote my new book I used Houdini when I wanted a very quick answer but I used Komodo for deep analysis.
You could also use a third program as a "tiebreaker" (assuming you have a quad) but if you use Critter it will usually side with Houdini, because they are both relatives of Ippo. For a totally uncorrelated third opinion use Stockfish.

[geots]

Just to clarify, I think you are saying that Komodo 4 gained more in blitz than in slow chess compared to Komodo 3. This could very well be true. However compared to Houdini, either Komodo is MUCH closer in slow chess than in blitz, just look at the CEGT data!

Hi Larry,

I have just purchased Komodo 4 and am enjoying it for analysis purposes!

My other main engine for post game analysis is Houdini 2 (don't worry I always go through my games on my own first before letting the engines loose )

When using Komodo 4 and Houdini in tandem what would you say their respective strengths and weaknesses are? Or to put it another way, where do they fill in each other's gaps?

In general my opinion is that Houdini will be more likely to be correct when there are hidden tactics in the position, but Komodo will be more likely to be correct when it's purely a question of evaluating positions. Houdini outrates Komodo on the lists primarily because Komodo spends more time in evaluation, and speed is usually decisive in engine vs engine play. This correlates with the notion that Houdini's opinion of the best move will more likely be correct after a few seconds but after some unknown amount of time Komodo should be more likely to be correct. The two programs are rather different so it's certainly good to see both opinions. When I wrote my new book I used Houdini when I wanted a very quick answer but I used Komodo for deep analysis.
You could also use a third program as a "tiebreaker" (assuming you have a quad) but if you use Critter it will usually side with Houdini, because they are both relatives of Ippo. For a totally uncorrelated third opinion use Stockfish.

Larry, please elaborate on the relationship between Critter and the Ippos. And do you consider Critter to be as close a relative as Houdini.

Keeping in mind that I am not expressing here anything at all about my feelings in this area. I am curious as to your feelings.



Best,

george

[lkaufman]

Larry, please elaborate on the relationship between Critter and the Ippos. And do you consider Critter to be as close a relative as Houdini.

Keeping in mind that I am not expressing here anything at all about my feelings in this area. I am curious as to your feelings.

Best,

george

Critter is most definitely NOT as close a relative to the Ippos as Houdini is. Houdini began life as a true clone of one of the Ippos, and has added important improvements without changing its nature. Critter on the other hand is a program actually written by its nominal author. It follows the Ippos in most respects, using all the same pruning formulas for example, but at least it does not use Ippo code. There are quite a few original ideas in Critter on top of those found in Ippo. So I would describe Houdini as a "brother" of Ivanhoe (or any Ippo), while I would describe Critter as a "cousin" of them.