Komodo 9.1 vs Stockfish 15061716 - 16 threads

[fastgm]

Dual AMD Opteron 6376

Komodo 9.1 vs Stockfish 15061716 - 16 threads
2048 MB Hash
TC = 20 minutes + 12 seconds

Code: Select all

 
    Program                   Elo    +   -   Games   Score   Av.Op.  Draws
 ---------------------------------------------------------------------------
  1 Komodo 9.1 T16          : 3018   20  19   326    55.2 %   2982   72.4 %
  2 Stockfish 15061716 T16  : 2982   19  20   326    44.8 %   3018   72.4 %


Wins   = 62
Draws  = 236
Losses = 28
Av.Op. Elo = 3000
 
Result     : 180.0/326 (+62,=236,-28)
Perf.      : 55.2 %
Margins    :
 68 %      : (+  1.4,-  1.4 %) -> [ 53.8, 56.6 %]
 95 %      : (+  2.8,-  2.8 %) -> [ 52.4, 58.0 %]
 99.7 %    : (+  4.3,-  4.2 %) -> [ 51.0, 59.5 %]
 
Elo        : 3036
Margins    :
 68 %      : (+ 10,- 10) -> [3026,3046]
 95 %      : (+ 20,- 19) -> [3017,3056]
 99.7 %    : (+ 30,- 29) -> [3007,3067]


Games        :    326 (finished)
 
White Wins   :     67 (20.6 %)
Black Wins   :     23 ( 7.1 %)
Draws        :    236 (72.4 %)
 
White Perf.  : 56.7 %
Black Perf.  : 43.3 %
 

Individual statistics:
 
1 Komodo 9.1 T16          : 3018  326 (+ 62,=236,- 28), 55.2 %
2 Stockfish 15061716 T16  : 2982  326 (+ 28,=236,- 62), 44.8 %

Games:
http://www.fastgm.de/schach/results-K91 ... 200+12.zip

[cdani]

Very interesting!! Thanks!

[Adam Hair]

Thanks for sharing this, Andreas.

[JJJ]

+36 ELo vs one of the lastest Stockfish Dev at this time control and number of CPU. I m not surprised at all.

[Laskos]

Thanks Andreas, excellent test as usual from you. The result is conclusive: Komodo 9.1 is the strongest engine by a significant margin on many cores and larger than blitz time controls.

[syzygy]

I would be curious to see the results under these conditions between Stockfish with 16 threads and Stockfish with 8 threads.

[lucasart]

Thanks Andreas, excellent test as usual from you. The result is conclusive: Komodo 9.1 is the strongest engine by a significant margin on many cores and larger than blitz time controls.

Yes, seems conclusive indeed. I'm expecting TCEC will be just like the previous one: SF vs. K in the final, and K wins (with > 80% draw rate).

But is it SMP scaling ? TC scaling ? or both ?

• TC scaling: I think it is well proven by now that K scales better than SF, as TC increases, SMP aside (1 core only).
• SMP scaling? Even if SMP scaling was the same, increasing the number of cores for equal TC is equivalent to increasing the TC. You always measure SMP+TC scaling, never SMP scaling separetely. So it's hard to conclude whether K's SMP scaling is really better, or if it's a side effect of TC scaling.

Unfortunately, none of that brings us any closer to writing SF patches that improve either TC scaling or SMP scaling

[Laskos]

Thanks Andreas, excellent test as usual from you. The result is conclusive: Komodo 9.1 is the strongest engine by a significant margin on many cores and larger than blitz time controls.

Yes, seems conclusive indeed. I'm expecting TCEC will be just like the previous one: SF vs. K in the final, and K wins (with > 80% draw rate).

But is it SMP scaling ? TC scaling ? or both ?

• TC scaling: I think it is well proven by now that K scales better than SF, as TC increases, SMP aside (1 core only).
• SMP scaling? Even if SMP scaling was the same, increasing the number of cores for equal TC is equivalent to increasing the TC. You always measure SMP+TC scaling, never SMP scaling separetely. So it's hard to conclude whether K's SMP scaling is really better, or if it's a side effect of TC scaling.

Unfortunately, none of that brings us any closer to writing SF patches that improve either TC scaling or SMP scaling

My impression (just an impression) is that it's both, mildly. A bit better SMP scaling and a bit better TC scaling. Also, from the tests of Andreas Strangmüller, Leto Atreides and Andrey Chilantiev, it seems that the expected score in 64 TCEC games in superfinal with 80% draw rate is +9 -4 for Komodo, with winning probability somewhere around 90%. I don't think SF can catch up until August TCEC.

[JJJ]

That's nice to read when you read before than Komdo team wouldn't catch up Stockfish team

[Laskos]

That's nice to read when you read before than Komdo team wouldn't catch up Stockfish team

Well, there were some people stating that, and who would have expected that Komodo team comes in less than two months after the release of Komodo 9.0 with a well scaling on multiple cores and long TC improvement of some 40 ELO points? This is pretty much a killer in TCEC conditions. It would have been much "easier" to have 40 ELO points improvement on 1 core at ultra-bullet, scaling to some 15 points at longer TC and multicore.