Komodo 4 on long time control

[lkaufman]

Just a comment as a prospective engine purchaser, I would really
like to see real position examples showing improvements on engine
web pages. Endless ELO statistics can get tiring, and tells you
nothing about analysis capabilities!!

Andrew

We could do this, and perhaps in the future we will, but basically it is misleading unless it is done to illustrate some specific piece of new knowledge added. Most improvement comes from two sources: improved search, which shows up in reduced time to "see" things, and more accurate eval, which means a higher frequency of "correct" evaluations. In both cases it's easy to find positions where any version looks better than another one, even if there are more positions where it would look worse. I'm sure we could find positions where Fritz 5.32 finds a better move than Komodo or Houdini if we looked hard enough. Some people like to show performance on a large set of problems, which at least correlates fairly well with improved rating but which can also be misleading, as some search changes help tactics but hurt positional play.

I'm referring primarily to comparing versions of the same program which
as you said may refer to new knowledge added or could be showing
bugs that have been fixed. Or even bugs that have yet to be fixed
The point is not the rating (there are many lists for that) but to show
how a program is improving for analysis. Some of the changes may have
a negligible affect on rating, others a huge affect!
Unfortunately in the past with most program updates the descriptions
given are often very wooly and vague, especially when they
come from Chessbase!

Andrew

It's not easy to explain the changes from one version to the next. It's not a matter of secrecy, rather it's just that if we changed a lot of the positional weights, as we have done recently, the effects are very hard to describe. Maybe our recent weight changes produce better evaluations 55% of the time and worse ones 45% of the time (just a guess). Obviously we could find examples where they helped, but this would be true even if we hurt the play 95% of the time.

[Werewolf]

Larry,
Is there one area where you've done a bit more work than any other like: "King Safety" or "rook and pawn endings" etc?


By the way I know your idea of King Safety in Rybka 3 was quite different to the Rybka 4 version - have you made any advancements on this?

[lkaufman]

Larry,
Is there one area where you've done a bit more work than any other like: "King Safety" or "rook and pawn endings" etc?


By the way I know your idea of King Safety in Rybka 3 was quite different to the Rybka 4 version - have you made any advancements on this?

Komodo's king safety is unique and not particularly similar to Rybka or anyone else's. We did a lot of work on it, but we have not made major changes to it recently. Most recently we have retuned all the evaluation weights. I've never been happy with the basic piece values in Rybka. In Komodo it was better, but finally now (with unreleased version) I think I finally got this about right. Current values agree with GM opinion in most materially unbalanced positions while also testing very well.

[Don]

Just a comment as a prospective engine purchaser, I would really
like to see real position examples showing improvements on engine
web pages. Endless ELO statistics can get tiring, and tells you
nothing about analysis capabilities!!

Andrew

I don't really have the talent for hyping changes, to make each change sound really exciting when in fact what we do is just plain hard work and most of it is pretty mundane. Only a few of our changes are worth writing home about but they add up to a lot of good chess.

[Lion]

Hi Don,
How close are we to a release of the MP version ?

[beram]

Hi there,

Perhaps this data out of Franks database is statistically more relevant.
I would take my hat off for Don Dailey when he can improve Komodo 4 with more than 60 ELO. I will do the same when he gains - more likely to me - an ELO gain of about 20-30 ELO.
The gain of Komodo 3 to Komodo 2 was 46 ELO and I doubt Don will top that because of the tendency of diminishing gains with each new release.

kind regards to both of you Don and Robert

Code: Select all

SCCT Auto232 Rating  2011
	
1	Houdini 2.0 Pro x64 1c	+84/=94/-46	58.48%	131.0/224
2	Komodo 3.0 x64 1c	+46/=94/-84	41.52%      93.0/224

According to latest publishing of Larry Kaufman on the Rybka forum today, 20 ELO is the expected ELO gain of Komodo 4. So still hat of for you Don, but take care with claims of overtaking Houdini 2.0

grt Bram

[MM]

Hi there,

Perhaps this data out of Franks database is statistically more relevant.
I would take my hat off for Don Dailey when he can improve Komodo 4 with more than 60 ELO. I will do the same when he gains - more likely to me - an ELO gain of about 20-30 ELO.
The gain of Komodo 3 to Komodo 2 was 46 ELO and I doubt Don will top that because of the tendency of diminishing gains with each new release.

kind regards to both of you Don and Robert

Code: Select all

SCCT Auto232 Rating  2011
	
1	Houdini 2.0 Pro x64 1c	+84/=94/-46	58.48%	131.0/224
2	Komodo 3.0 x64 1c	+46/=94/-84	41.52%      93.0/224

According to latest publishing of Larry Kaufman on the Rybka forum today, 20 ELO is the expected ELO gain of Komodo 4. So still hat of for you Don, but take care with claims of overtaking Houdini 2.0

grt Bram

Hi, can you publish the link in which Larry said so, please?

Thank you

Regards

[Don]

Hi there,

Perhaps this data out of Franks database is statistically more relevant.
I would take my hat off for Don Dailey when he can improve Komodo 4 with more than 60 ELO. I will do the same when he gains - more likely to me - an ELO gain of about 20-30 ELO.
The gain of Komodo 3 to Komodo 2 was 46 ELO and I doubt Don will top that because of the tendency of diminishing gains with each new release.

kind regards to both of you Don and Robert

Code: Select all

SCCT Auto232 Rating  2011
	
1	Houdini 2.0 Pro x64 1c	+84/=94/-46	58.48%	131.0/224
2	Komodo 3.0 x64 1c	+46/=94/-84	41.52%      93.0/224

According to latest publishing of Larry Kaufman on the Rybka forum today, 20 ELO is the expected ELO gain of Komodo 4. So still hat of for you Don, but take care with claims of overtaking Houdini 2.0

grt Bram

Hi, can you publish the link in which Larry said so, please?

Thank you

Regards

We have found it difficult to predict the strength of release versions. We have seriously understated the rating at times and at other times seriously overstanted it. And as you can see we are not the only ones, it just shows that it is difficult to determine how any given program will perform once it's released and tested at many time controls and hardware combinations.

All we know is that there is a significant improvement, but it's nothing like the 100 ELO we got a couple of versions ago. My best lower bound is 20 ELO. In our fast testing it is around 35 ELO. As they say, YMMV.

It won't be up to Houdini's strength yet - my time has been seriously divided in the last 2 or 3 months with personal responsibilities, trips away, MP work, and my illness. When the MP version is released we will be able to completely focus on strength increases and have many ideas.

[MM]

Hi there,

Perhaps this data out of Franks database is statistically more relevant.
I would take my hat off for Don Dailey when he can improve Komodo 4 with more than 60 ELO. I will do the same when he gains - more likely to me - an ELO gain of about 20-30 ELO.
The gain of Komodo 3 to Komodo 2 was 46 ELO and I doubt Don will top that because of the tendency of diminishing gains with each new release.

kind regards to both of you Don and Robert

Code: Select all

SCCT Auto232 Rating  2011
	
1	Houdini 2.0 Pro x64 1c	+84/=94/-46	58.48%	131.0/224
2	Komodo 3.0 x64 1c	+46/=94/-84	41.52%      93.0/224

According to latest publishing of Larry Kaufman on the Rybka forum today, 20 ELO is the expected ELO gain of Komodo 4. So still hat of for you Don, but take care with claims of overtaking Houdini 2.0

grt Bram

Hi, can you publish the link in which Larry said so, please?

Thank you

Regards

We have found it difficult to predict the strength of release versions. We have seriously understated the rating at times and at other times seriously overstanted it. And as you can see we are not the only ones, it just shows that it is difficult to determine how any given program will perform once it's released and tested at many time controls and hardware combinations.

All we know is that there is a significant improvement, but it's nothing like the 100 ELO we got a couple of versions ago. My best lower bound is 20 ELO. In our fast testing it is around 35 ELO. As they say, YMMV.

It won't be up to Houdini's strength yet - my time has been seriously divided in the last 2 or 3 months with personal responsibilities, trips away, MP work, and my illness. When the MP version is released we will be able to completely focus on strength increases and have many ideas.

Hello Mr Dailey,

Thank you for the explaination.
No intention to hurry you up of course, considering this situation wouldn't be helpful to release a SP version available for all in order to receive advices from the users and get new ideas to improve the engine?

Many users like to give feedback' in order to help the programmers to improve and refine the engine.

Best Regards

[Don]

Hi there,

Perhaps this data out of Franks database is statistically more relevant.
I would take my hat off for Don Dailey when he can improve Komodo 4 with more than 60 ELO. I will do the same when he gains - more likely to me - an ELO gain of about 20-30 ELO.
The gain of Komodo 3 to Komodo 2 was 46 ELO and I doubt Don will top that because of the tendency of diminishing gains with each new release.

kind regards to both of you Don and Robert

Code: Select all

SCCT Auto232 Rating  2011
	
1	Houdini 2.0 Pro x64 1c	+84/=94/-46	58.48%	131.0/224
2	Komodo 3.0 x64 1c	+46/=94/-84	41.52%      93.0/224

According to latest publishing of Larry Kaufman on the Rybka forum today, 20 ELO is the expected ELO gain of Komodo 4. So still hat of for you Don, but take care with claims of overtaking Houdini 2.0

grt Bram

Hi, can you publish the link in which Larry said so, please?

Thank you

Regards

We have found it difficult to predict the strength of release versions. We have seriously understated the rating at times and at other times seriously overstanted it. And as you can see we are not the only ones, it just shows that it is difficult to determine how any given program will perform once it's released and tested at many time controls and hardware combinations.

All we know is that there is a significant improvement, but it's nothing like the 100 ELO we got a couple of versions ago. My best lower bound is 20 ELO. In our fast testing it is around 35 ELO. As they say, YMMV.

It won't be up to Houdini's strength yet - my time has been seriously divided in the last 2 or 3 months with personal responsibilities, trips away, MP work, and my illness. When the MP version is released we will be able to completely focus on strength increases and have many ideas.

Hello Mr Dailey,

Thank you for the explaination.
No intention to hurry you up of course, considering this situation wouldn't be helpful to release a SP version available for all in order to receive advices from the users and get new ideas to improve the engine?

Many users like to give feedback' in order to help the programmers to improve and refine the engine.

Best Regards

We like to get feedback too. I would prefer anyone sends feeback directly to me, my email is not hidden - you can find it in my profile. If the comment is of general interest you can post it here too but give me a heads up as I don't always keep up with this forum.