CCRL 40/40 lists updated (8th June 2013)

[Laskos]

Houdini tactical has only about 30% of the number of games that Houdini 3 has, so it is far more likely that Houdini tactical is overrated than that Houdini 3 is underrated significantly. Also, the margin of error for H3 is shown as 15. The most likely thing is that H3 is a couple elo underrated on that list, so let's say we need 25 elo to be favored to lead the list.

Averaging over H3 and H3 T (25-30 points weaker) gives that H3 is underrated by some 8 Elo points. So the difference between H3 and Komodo CCT is more likely 30+/-20 Elo points 95% confidence.

But your claim that H3 T is 25-30 points weaker is presumably based on the blitz lists. At higher levels all the rating differences contract, and it may well be that there is very little difference between the versions at 40/40 level, since sometimes different programs/versions benefit more than others from more time. The best evidence of the difference is the actual testing at this level. I admit it is unlikely that there is no difference, but maybe 10 elo might be a good guess, in which case H3 would be about 3 elo underrated.

Well, by your reasoning, Houdini 3 Tactical is an outstanding engine, being only 10 points weaker than the strongest setting of H3, scaling well, and performing exceptionally well (best by far) in test suites. I remember Rybka WinFinder being 200 points weaker than the Rybka default.

[Don]

So, we just have to "find" 22 elo points to catch Houdini 3 at 40/40. I wonder how long that will take us...

If I were you and don I would not be so concerned with Houdini.
I have been testing a development version of stockfish. It has achieved a equal rating with Houdini 3 pro at long time controls with a error bar of +\- 20 Elo.

I started testing a new development version of stockfish yesterday at 40/40 that could be even stronger. Will have a update on this version of stockfish in a week or two. Why the concern for you....stockfish is free and making progress.

Stockfish has always been my favorite chess program behind Komodo of course. So I am thrilled that it is continuing to improve, especially after the news a while back that development had almost stopped. I think we now have a very nice rivalry at the top with 3 unrelated engine families. The Glaurung/Stockfish family, the Doch/Komodo family and the Ippo/Houdini family.

[mwyoung]

So, we just have to "find" 22 elo points to catch Houdini 3 at 40/40. I wonder how long that will take us...

If I were you and don I would not be so concerned with Houdini.
I have been testing a development version of stockfish. It has achieved a equal rating with Houdini 3 pro at long time controls with a error bar of +\- 20 Elo.

I started testing a new development version of stockfish yesterday at 40/40 that could be even stronger. Will have a update on this version of stockfish in a week or two. Why the concern for you....stockfish is free and making progress.

Stockfish has always been my favorite chess program behind Komodo of course. So I am thrilled that it is continuing to improve, especially after the news a while back that development had almost stopped. I think we now have a very nice rivalry at the top with 3 unrelated engine families. The Glaurung/Stockfish family, the Doch/Komodo family and the Ippo/Houdini family.

Yes, it is great to see competition. This can only help the computer chess world.

BTW I want to thank You and Larry for giving your results against other chess engines. Being good, or bad, or incomplete for Komodo MP. That is a rare thing to see in the computer chess world for a commercial chess program that has not been released.