Komodo 5 64-bit for CCRL 40/4

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Komodo 5 64-bit for CCRL 40/4

Post by Adam Hair »

New update:

Added Quazar 0.4, Booot 5.1.0, Gull 1.1 (its results are being combined with Gull 1.0a for this estimate), Nemo 1.0.1b, Zappa Mexico II, Strelka 5.1, and Protector 1.4.0 as opponents.

The improvement over Komodo 4 is back down to 33 to 36 Elo.

Bayeselo (default)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3208   21   20   910   70%  3054   31% 
   2 Komodo 5 64-bit                     3165   27   26   574   72%  2990   27% 
   3 Strelka 5.1 64-bit                  3158   24   24   569   66%  3042   43% 
   4 Critter 1.4 64-bit                  3147   19   19  1004   67%  3022   40% 
   5 Komodo 4 64-bit                     3132   17   17  1395   69%  2984   30% 
   6 Stockfish 2.2.2 64-bit              3127   20   20   834   59%  3060   39% 
   7 Rybka 4.1 64-bit                    3123   17   16  1365   66%  3002   36% 
   8 Fritz 13                            2989   17   17  1284   63%  2885   30% 
   9 Naum 4.2 64-bit                     2981   17   17  1183   50%  2979   35% 
  10 Chiron 1.1a 64-bit                  2979   20   20   890   62%  2892   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  2965   20   19   907   57%  2919   37% 
  12 Hannibal 1.2 64-bit                 2945   22   22   715   50%  2947   35% 
  13 Hiarcs 13.2                         2928   22   22   714   52%  2920   35% 
  14 Gull 1.0a 64-bit                    2928   18   17  1383   63%  2827   27% 
  15 Spark 1.0 64-bit                    2920   16   15  1528   51%  2915   34% 
  16 Spike 1.4 Leiden                    2918   18   17  1206   51%  2915   33%
Bayeselo (prior 0.1, mm 1 1, scale 1)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3271   22   22   910   70%  3106   31% 
   2 Komodo 5 64-bit                     3226   29   28   574   72%  3039   27% 
   3 Strelka 5.1 64-bit                  3217   26   26   569   66%  3094   43% 
   4 Critter 1.4 64-bit                  3206   20   20  1004   67%  3073   40% 
   5 Komodo 4 64-bit                     3190   18   18  1395   69%  3032   30% 
   6 Stockfish 2.2.2 64-bit              3185   22   21   834   59%  3114   39% 
   7 Rybka 4.1 64-bit                    3181   18   18  1365   66%  3052   36% 
   8 Fritz 13                            3038   19   19  1284   63%  2926   30% 
   9 Naum 4.2 64-bit                     3029   19   18  1183   50%  3027   35% 
  10 Chiron 1.1a 64-bit                  3027   22   22   890   62%  2933   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  3011   21   21   907   57%  2963   37% 
  12 Hannibal 1.2 64-bit                 2991   24   24   715   50%  2993   35% 
  13 Hiarcs 13.2                         2973   24   24   714   52%  2963   35% 
  14 Gull 1.0a 64-bit                    2972   19   19  1383   63%  2865   27% 
  15 Spark 1.0 64-bit                    2964   17   17  1528   51%  2958   34% 
  16 Spike 1.4 Leiden                    2962   19   19  1206   51%  2959   33%
Ordo (with White advantage)

Code: Select all

                            ENGINE:  RATING    POINTS  PLAYED    (%)
               Houdini 2.0c 64-bit:  3274.3     640.0     910   70.3%
                   Komodo 5 64-bit:  3224.8     417.5     577   72.4%
                Strelka 5.1 64-bit:  3223.6     376.0     569   66.1%
                Critter 1.4 64-bit:  3214.5     670.5    1004   66.8%
                   Komodo 4 64-bit:  3191.1     962.5    1395   69.0%
            Stockfish 2.2.2 64-bit:  3189.7     493.5     834   59.2%
                  Rybka 4.1 64-bit:  3186.6     905.0    1365   66.3%
                          Fritz 13:  3032.8     812.5    1284   63.3%
                   Naum 4.2 64-bit:  3025.6     595.0    1183   50.3%
                Chiron 1.1a 64-bit:  3022.9     552.0     890   62.0%
Deep Shredder 12 64-bit OA On 1CPU:  3007.2     512.5     907   56.5%
               Hannibal 1.2 64-bit:  2987.1     358.0     715   50.1%
                       Hiarcs 13.2:  2970.1     370.0     714   51.8%
                  Gull 1.0a 64-bit:  2968.8     866.5    1383   62.7%
                  Spark 1.0 64-bit:  2960.3     785.0    1528   51.4%
                  Spike 1.4 Leiden:  2955.9     612.5    1206   50.8%
Top
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Komodo 5 64-bit for CCRL 40/4

Post by Adam Hair »

Probably the next-to-last update from me. The increase over Komodo 4 is 33 to 34 Elo for my conditions (at this point):

Code: Select all

Komodo 5 64-bit               : 730 (+431,=199,-100), 72.7 %

Rybka 4.1 64-bit              :  52 (+ 18,= 26,-  8), 59.6 %
Hannibal 1.2 64-bit           :  52 (+ 37,= 12,-  3), 82.7 %
Houdini 2.0c 64-bit           :  52 (+ 10,= 15,- 27), 33.7 %
Naum 4.2 64-bit               :  52 (+ 38,= 10,-  4), 82.7 %
Critter 1.4 64-bit            :  48 (+ 13,= 21,- 14), 49.0 %
Spike 1.4 Leiden              :  48 (+ 33,= 11,-  4), 80.2 %
Spark 1.0 64-bit              :  48 (+ 33,= 11,-  4), 80.2 %
Stockfish 2.2.2 64-bit        :  48 (+ 18,= 21,-  9), 59.4 %
Gull 1.0a 64-bit              :  92 (+ 61,= 23,-  8), 78.8 %
Booot 5.1.0                   :  44 (+ 40,=  3,-  1), 94.3 %
Nemo 1.0.1b 64-bit            :  44 (+ 36,=  7,-  1), 89.8 %
Protector 1.4.0 64-bit        :  43 (+ 33,=  6,-  4), 83.7 %
Quazar 0.4 64-bit             :  39 (+ 29,=  9,-  1), 85.9 %
Strelka 5.1 64-bit            :  36 (+  8,= 17,- 11), 45.8 %
Zappa Mexico II 64-bit        :  32 (+ 24,=  7,-  1), 85.9 %

Bayeselo (default)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3208   20   20   920   70%  3055   32% 
   2 Komodo 5 64-bit                     3163   24   23   730   73%  2988   27% 
   3 Strelka 5.1 64-bit                  3156   24   24   581   65%  3044   43% 
   4 Critter 1.4 64-bit                  3147   19   19  1012   67%  3023   40% 
   5 Komodo 4 64-bit                     3132   17   17  1395   69%  2985   30% 
   6 Stockfish 2.2.2 64-bit              3127   20   20   842   59%  3061   40% 
   7 Rybka 4.1 64-bit                    3123   17   16  1373   66%  3003   36% 
   8 Fritz 13                            2989   18   17  1284   63%  2885   30% 
   9 Naum 4.2 64-bit                     2980   17   17  1195   50%  2981   35% 
  10 Chiron 1.1a 64-bit                  2979   20   20   890   62%  2892   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  2965   20   19   907   57%  2919   37% 
  12 Hannibal 1.2 64-bit                 2945   22   22   723   50%  2950   35% 
  13 Gull 1.0a 64-bit                    2930   17   17  1403   62%  2832   27% 
  14 Hiarcs 13.2                         2929   22   22   714   52%  2920   35% 
  15 Spark 1.0 64-bit                    2920   15   15  1536   51%  2916   34% 
  16 Spike 1.4 Leiden                    2919   18   17  1214   51%  2917   33%

Bayeselo (prior 0.1, mm 1 1, scale 1)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3271   22   22   920   70%  3107   32% 
   2 Komodo 5 64-bit                     3224   26   25   730   73%  3036   27% 
   3 Strelka 5.1 64-bit                  3215   26   26   581   65%  3096   43% 
   4 Critter 1.4 64-bit                  3206   20   20  1012   67%  3074   40% 
   5 Komodo 4 64-bit                     3190   18   18  1395   69%  3032   30% 
   6 Stockfish 2.2.2 64-bit              3185   22   21   842   59%  3115   40% 
   7 Rybka 4.1 64-bit                    3180   18   18  1373   66%  3053   36% 
   8 Fritz 13                            3038   19   19  1284   63%  2926   30% 
   9 Naum 4.2 64-bit                     3028   18   19  1195   50%  3029   35% 
  10 Chiron 1.1a 64-bit                  3027   22   22   890   62%  2933   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  3011   21   21   907   57%  2963   37% 
  12 Hannibal 1.2 64-bit                 2990   23   23   723   50%  2995   35% 
  13 Gull 1.0a 64-bit                    2974   19   19  1403   62%  2870   27% 
  14 Hiarcs 13.2                         2973   24   24   714   52%  2963   35% 
  15 Spark 1.0 64-bit                    2964   17   17  1536   51%  2960   34% 
  16 Spike 1.4 Leiden                    2962   19   19  1214   51%  2960   33%

Ordo (with White advantage)

Code: Select all

                            ENGINE:  RATING    POINTS  PLAYED    (%)
               Houdini 2.0c 64-bit:  3274.4     646.5     920   70.3%
                   Komodo 5 64-bit:  3224.0     532.0     733   72.6%
                Strelka 5.1 64-bit:  3221.0     380.5     581   65.5%
                Critter 1.4 64-bit:  3214.0     674.5    1012   66.7%
                   Komodo 4 64-bit:  3190.6     962.5    1395   69.0%
            Stockfish 2.2.2 64-bit:  3188.7     496.5     842   59.0%
                  Rybka 4.1 64-bit:  3185.8     908.0    1373   66.1%
                          Fritz 13:  3032.6     812.5    1284   63.3%
                   Naum 4.2 64-bit:  3024.3     596.5    1195   49.9%
                Chiron 1.1a 64-bit:  3022.6     552.0     890   62.0%
Deep Shredder 12 64-bit OA On 1CPU:  3006.7     512.5     907   56.5%
               Hannibal 1.2 64-bit:  2986.6     359.5     723   49.7%
                  Gull 1.0a 64-bit:  2969.9     872.0    1403   62.2%
                       Hiarcs 13.2:  2969.7     370.0     714   51.8%
                  Spark 1.0 64-bit:  2959.9     786.0    1536   51.2%
                  Spike 1.4 Leiden:  2956.2     614.5    1214   50.6%
Top
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Komodo 5 64-bit for CCRL 40/4

Post by Adam Hair »

Final results for my testing. The improvement over Komodo 4 64-bit is 30 to 31 Elo, given my conditions and opponents. There are several other opponents that my testing does not cover (Shredder, Hiarcs, Chiron, Sjeng, Fritz).

CPU : Intel E8400
OS : Windows XP 64-bit
Time Control : 40 moves in 3 minutes, repeating (Conforms to CCRL 40/4 reference)
Openings : a pgn of ~18,000 games, truncated to 4 full moves, all duplicates removed. Positions are not repeated
CPUs : 1
Hash : 128 MB
GUI : cutechess-cli

Bayeselo (default)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3209   20   20   932   70%  3056   32% 
   2 Komodo 5 64-bit                     3160   22   21   885   72%  2992   28% 
   3 Strelka 5.1 64-bit                  3156   24   24   585   65%  3045   43% 
   4 Critter 1.4 64-bit                  3148   19   18  1028   67%  3025   40% 
   5 Komodo 4 64-bit                     3132   17   17  1395   69%  2985   30% 
   6 Stockfish 2.2.2 64-bit              3125   20   20   858   58%  3063   40% 
   7 Rybka 4.1 64-bit                    3123   16   16  1385   66%  3005   36% 
   8 Fritz 13                            2989   17   17  1284   63%  2885   30% 
   9 Naum 4.2 64-bit                     2980   17   17  1207   50%  2983   35% 
  10 Chiron 1.1a 64-bit                  2980   20   20   890   62%  2892   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  2965   20   19   907   57%  2919   37% 
  12 Hannibal 1.2 64-bit                 2948   22   22   735   50%  2953   35% 
  13 Hiarcs 13.2                         2929   22   22   714   52%  2920   35% 
  14 Gull 1.0a 64-bit                    2927   17   17  1423   61%  2837   27% 
  15 Spark 1.0 64-bit                    2921   15   15  1552   51%  2919   34% 
  16 Spike 1.4 Leiden                    2918   17   17  1230   50%  2920   33%
Bayeselo (prior 0.1, mm 1 1, scale 1)

Code: Select all

Rank Name                                 Elo    +    - games score oppo. draws 
   1 Houdini 2.0c 64-bit                 3272   22   21   932   70%  3108   32% 
   2 Komodo 5 64-bit                     3220   23   23   885   72%  3040   28% 
   3 Strelka 5.1 64-bit                  3216   26   26   585   65%  3097   43% 
   4 Critter 1.4 64-bit                  3207   20   20  1028   67%  3076   40% 
   5 Komodo 4 64-bit                     3190   18   18  1395   69%  3032   30% 
   6 Stockfish 2.2.2 64-bit              3183   21   21   858   58%  3116   40% 
   7 Rybka 4.1 64-bit                    3180   18   18  1385   66%  3054   36% 
   8 Fritz 13                            3038   19   19  1284   63%  2926   30% 
   9 Naum 4.2 64-bit                     3028   18   18  1207   50%  3031   35% 
  10 Chiron 1.1a 64-bit                  3027   22   22   890   62%  2934   32% 
  11 Deep Shredder 12 64-bit OA On 1CPU  3011   21   21   907   57%  2963   37% 
  12 Hannibal 1.2 64-bit                 2993   23   23   735   50%  2999   35% 
  13 Hiarcs 13.2                         2973   24   24   714   52%  2963   35% 
  14 Gull 1.0a 64-bit                    2971   19   19  1423   61%  2875   27% 
  15 Spark 1.0 64-bit                    2964   16   16  1552   51%  2962   34% 
  16 Spike 1.4 Leiden                    2962   19   19  1230   50%  2964   33%
Ordo (with White advantage)

Code: Select all

                            ENGINE:  RATING    POINTS  PLAYED    (%)
               Houdini 2.0c 64-bit:  3275.0     654.0     932   70.2%
                   Komodo 5 64-bit:  3222.0     636.0     885   71.9%
                Strelka 5.1 64-bit:  3221.8     383.0     585   65.5%
                Critter 1.4 64-bit:  3215.9     684.5    1028   66.6%
                   Komodo 4 64-bit:  3190.9     962.5    1395   69.0%
            Stockfish 2.2.2 64-bit:  3186.9     501.5     858   58.4%
                  Rybka 4.1 64-bit:  3186.0     913.5    1385   66.0%
                          Fritz 13:  3032.8     812.5    1284   63.3%
                   Naum 4.2 64-bit:  3024.5     599.5    1207   49.7%
                Chiron 1.1a 64-bit:  3023.0     552.0     890   62.0%
Deep Shredder 12 64-bit OA On 1CPU:  3007.1     512.5     907   56.5%
               Hannibal 1.2 64-bit:  2989.5     364.5     735   49.6%
                       Hiarcs 13.2:  2970.1     370.0     714   51.8%
                  Gull 1.0a 64-bit:  2967.8     873.0    1423   61.3%
                  Spark 1.0 64-bit:  2960.3     789.5    1552   50.9%
                  Spike 1.4 Leiden:  2955.6     616.5    1230   50.1%
Top

Return to “Computer Chess Club: Tournaments and Matches”