Made In Heaven class Time Control Comparison

[Aser Huerga]

As a Shaun Brewer suggestion, I decided to run my games at different Time Controls to see how the top engines strength change as time increases. Here are the results:

Five i7-3930K CPUs 4.25 GHz
1 core for all engines
Ponder off
1024 Hash
3-4-5 EGTBs (when available) in SSDs

Code: Select all

3'+1" Time Control

   # PLAYER          : RATING  ERROR   POINTS  PLAYED    (%)
   1 Houdini 4       :   32.1   13.3    340.0     600   56.7%
   2 Komodo TCEC     :  -14.4   13.4    282.0     600   47.0%
   3 Stockfish DD    :  -17.6   13.8    278.0     600   46.3%


9'+3" Time Control

   # PLAYER          : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish DD    :   16.6   14.1    320.5     600   53.4%
   2 Houdini 4       :    8.1   13.5    310.0     600   51.7%
   3 Komodo TCEC     :  -24.7   13.4    269.5     600   44.9%

27'+9" Time Control

   # PLAYER          : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish DD    :   22.6   14.1    328.0     600   54.7%
   2 Houdini 4       :    5.7   13.4    307.0     600   51.2%
   3 Komodo TCEC     :  -28.3   13.4    265.0     600   44.2%

54'+18" Time Control

   # PLAYER          : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish DD    :   11.4   14.2    314.0     600   52.3%
   2 Houdini 4       :    0.4   13.7    300.5     600   50.1%
   3 Komodo TCEC     :  -11.8   13.6    285.5     600   47.6%

90'+30" Time Control

   # PLAYER          : RATING  ERROR   POINTS  PLAYED    (%)
   1 Stockfish DD    :   10.5   12.9    313.0     600   52.2%
   2 Komodo TCEC     :    0.8   13.2    301.0     600   50.2%
   3 Houdini 4       :  -11.3   13.1    286.0     600   47.7%

I want to thanks Adam Hair for his help in the presentation of graph and results.

( All the games can be downloaded here: TTC_All_Games )

[Laskos]

Thanks, Aser. This confirms that SF doesn't need much to overtake Houdini, Komodo does need some time, and scales well afterwards.

[Stefan Schiffermueller]

Interesting. What is the value of the contempt-parameter in Houdini 4?

[Aser Huerga]

Interesting. What is the value of the contempt-parameter in Houdini 4?

Thanks Stefan.
All engines plays at default settings.

[PaulieD]

A picture truly is worth a thousand words. That is a wonderful depiction of what a lot of people have been trying to say.

[ouachita]

A picture truly is worth a thousand words. That is a wonderful depiction of what a lot of people have been trying to say.

Yep, I've posted numerous time that it looked to me that the crossover was >60+, and I actually worked on the same type of Excel chart. Then I went off on rants saying one cannot accurately extrapolate these 90+ data points out to 40/120, ad infinitum.

[ouachita]

However, we can feel safe in saying that:

1. it seems highly unlikely that H's Elo line would ever intersect the other two.
2. the SF and K lines >90 may or may not intersect.

[Shaun]

Thank you!!!

[Modern Times]

Brilliant work Aser, the graph is very enlightening.

Question is, at what point does Komodo level off...

[ouachita]

Question is, at what point does Komodo level off...

. . . which can only be answered thru LTC (120+) testing. Perhaps a day or day on the cluster would help to answer this question?