Komodo 5.1 MP results

[lkaufman]

Bear in mind that the underlying SP program is not as strong as CCT, and that MP is not yet refined. Considering this, we are not unhappy with 55% against SF3 on 4 cores.

[lkaufman]

Here are some results for what I believe will be the Komodo version that will be available Monday as Komodo 5.1 MP. These results are against Stockfish 3, all at 3" + 2" increment on sse4 machines. Noomen testsuite, 256 k hash. 200 games were played on 4 cores each, 100 on 12 cores each.
Total score: 164 -136, which is +33 elo. The results were actually slightly better on the 12 core machine than on the other two (57.5 to 42.5 on 12 core), which at least suggests that Komodo doesn't scale badly on big hardware.
Based on a private run of 1500 games by one tester, we estimate that on one core Komodo is 43 elo above Stockfish 3. So a lead of 33 on 4 or more cores is just slightly worse, within the margin of error. It would seem that Komodo MP gets close to the same benefit as does Stockfish from using MP.
Against Critter 1.6a on 4 cores, I have a +32 elo result after 53 games, which agrees well with the SF result as Critter 1.6a and SF3 are very close in strength.

[beram]

Here are some results for what I believe will be the Komodo version that will be available Monday as Komodo 5.1 MP. These results are against Stockfish 3, all at 3" + 2" increment on sse4 machines. Noomen testsuite, 256 k hash. 200 games were played on 4 cores each, 100 on 12 cores each.
Total score: 164 -136, which is +33 elo. The results were actually slightly better on the 12 core machine than on the other two (57.5 to 42.5 on 12 core), which at least suggests that Komodo doesn't scale badly on big hardware.
Based on a private run of 1500 games by one tester, we estimate that on one core Komodo is 43 elo above Stockfish 3. So a lead of 33 on 4 or more cores is just slightly worse, within the margin of error. It would seem that Komodo MP gets close to the same benefit as does Stockfish from using MP.
Against Critter 1.6a on 4 cores, I have a +32 elo result after 53 games, which agrees well with the SF result as Critter 1.6a and SF3 are very close in strength.

When you take a sample of the grand unified list for 4CPU 2013-05-12,
SF 3 4CPU is actually 14 - 24 ELO better than Critter 1.6 and 1.6a
Would Komodo 5.1 MP also be 33 Elo better than SF 3 4CPU on this list, it would surpass Houdini 2 with about 10 ELO
and become second with 3308. So there would remain a 38 ELO gap for first place. Still exciting to see for what will be the outcome

Code: Select all

Rnk Name                                Elo      +/-    Games  Score     Oppo.  Draws  
  1 Houdini 3.0 64-bit 4CPU          3346.97    9.50     4164  76.45%  3155.40  34.99% 

 2 Komodo 5.1 MP 64-bit 4CPU        3308 (estimated on testruns by LK)  

  3 Houdini 2.0 64-bit 4CPU          3297.44    9.80     3840  72.86%  3125.73  35.78% 
  4 Houdini 1.5 64-bit 4CPU          3290.21    9.74     3648  70.34%  3145.65  38.10% 
  5 Stockfish 3.0 64-bit 4CPU        3275.97   26.90      411  59.85%  3210.84  50.61% 
  6 Critter 1.6 64-bit 4CPU          3261.69   17.97      873  57.96%  3212.17  56.36% 
  7 Equinox 1.70 64-bit 4CPU         3260.80   13.36     1914  71.53%  3111.40  42.11% 
 10 Critter 1.2 64-bit 4CPU          3253.03    8.78     4546  70.27%  3103.76  40.45% 
 11 Stockfish 2.3 64-bit 4CPU        3252.30   17.70      898  55.79%  3217.87  55.46% 
 12 Critter 1.6a 64-bit 4CPU         3251.31   13.60     1519  59.02%  3196.57  58.13%