Survival of the Fittest 7

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

enhorning
Posts: 342
Joined: Wed Jan 05, 2011 10:05 pm

Survival of the Fittest 7

Post by enhorning »

These 11 programs qualified from last round:

Code: Select all

Rating - Program - Version - Cores
2954 Robbolito 0.21Q [4c]
2989 Gull 2.3 [4c]
3007 Gull 2.2 [4c]
3014 Stockfish 4 [4c]
3017 Stockfish 3 JA [4c]
3024 Komodo 5.1 [4c]
3037 Critter 1.6a [4c]
3041 Houdini 1.5a [4c]
3061 Stockfish 191113 [4c]
3076 Komodo 6 [4c]
3089 Houdini 4 [4c]
(Ratings are a normalized average of multiple rating lists, weighted together with bayeselo performance from previous rounds, and should only be seen as a rough guess.)

That's the 11 participating programs

The tournament is an eight-double round robin (440 games total), and time control will be 6 minutes + 4 seconds increment / move. The programs that support multiple cores are generally allowed to use 4 cores out of my 6 available. Hash is set fairly low - only 256 MB - for the same reason that 2 cores are unused: so that me using the computer while games are running won't interfere with the engines.

5 men Gaviota, Nalimov, Scorpio, Syzygy and 4 men Shredder TBs/BBs are installed and enabled for the programs that support them (except those where TBs were sometimes causing crashes).

I am using a file containing 57 short openings (max 8 plies, but often shorter): https://www.dropbox.com/s/ft3oee8fubdp3 ... ariety.pgn - this file was created by taking lines that were played in at least 0.5% of games between 2600+ players (using the ChessTempo database), and for which Aquarium's CAP evaluation was within [0.00 - 0.22], i.e. up to 3 steps away from the root value. So, my hope is that this would give lines that are playable both according to human and engine opinion.

Suggestion for programs to include in future tournaments are always welcome My plan is that this round will be the last in this series. After that, I'll start over with lower rated programs again - as a guideline, I'd like to include programs with CCRL ratings at least around 2540 or CEGT ratings at least around 2400.
enhorning
Posts: 342
Joined: Wed Jan 05, 2011 10:05 pm

Re: Survival of the Fittest 7

Post by enhorning »

Standings after 1 double round-robin cycle:

Code: Select all

    Engine                Score   St Ho Ho Ko St Gu Ko Cr Ro Gu St    S-B
01: Stockfish 191113 [4c] 13.0/20 ·· == 11 == 0= =0 1= 11 == 11 =1  122.25
02: Houdini 4 [4c]        12.0/20 == ·· =0 10 == == == 11 11 0= 11  111.00
03: Houdini 1.5a [4c]     12.0/20 00 =1 ·· == == 10 == =1 11 1= =1  109.50
04: Komodo 5.1 [4c]       11.5/20 == 01 == ·· =1 =0 == == == =1 11  108.75
05: Stockfish 3 JA [4c]   11.0/20 1= == == =0 ·· 0= == 01 =1 1= 1=  105.00
06: Gull 2.3 [4c]         10.5/20 =1 == 01 =1 1= ·· =0 =0 == == ==  107.75
07: Komodo 6 [4c]         10.5/20 0= == == == == =1 ·· 0= 11 == ==  101.50
08: Critter 1.6a [4c]     8.5/20  00 00 =0 == 10 =1 1= ·· 10 == ==   81.00
09: RobboLito 0.21Q [4c]  7.5/20  == 00 00 == =0 == 00 01 ·· =1 =1   69.25
10: Gull 2.2 [4c]         7.0/20  00 1= 0= =0 0= == == == =0 ·· =0   71.75
11: Stockfish 4 [4c]      6.5/20  =0 00 =0 00 0= == == == =0 =1 ··   61.75

110 of 440 games played
PGN with games is at: https://www.dropbox.com/s/praslvil9sqn7 ... al%207.pgn
enhorning
Posts: 342
Joined: Wed Jan 05, 2011 10:05 pm

Re: Survival of the Fittest 7

Post by enhorning »

Standings at the halfway point:

Code: Select all

    Engine                Score     Ho   St   Ko   Gu   Ko   St   Ho   Cr   Gu   St   Ro    S-B
01: Houdini 4 [4c]        26.5/40 ···· ===1 10== ===1 ==1= ==1= =011 111= 0=01 11== 1111  505.25
02: Stockfish 191113 [4c] 24.5/40 ===0 ···· ==10 =0=1 1=== 0=== 11=1 1110 111= =1== ==1=  467.25
03: Komodo 5.1 [4c]       22.0/40 01== ==01 ···· =00= ===1 =1=0 ==01 ==1= =11= 11== ====  427.50
04: Gull 2.3 [4c]         21.0/40 ===0 =1=0 =11= ···· =0=0 1=== 0111 =0== ==== ==== ==1=  414.00
05: Komodo 6 [4c]         20.0/40 ==0= 0=== ===0 =1=1 ···· ==1= ===1 0==0 ===0 ==== 11==  391.50
06: Stockfish 3 JA [4c]   19.5/40 ==0= 1=== =0=1 0=== ==0= ···· ==== 01== 1==0 1=== =1=0  387.50
07: Houdini 1.5a [4c]     19.5/40 =100 00=0 ==10 1000 ===0 ==== ···· =1=0 1=1= =1=1 1110  368.00
08: Critter 1.6a [4c]     19.0/40 000= 0001 ==0= =1== 1==1 10== =0=1 ···· ===1 ==0= 101=  365.25
09: Gull 2.2 [4c]         17.0/40 1=10 000= =00= ==== ===1 0==1 0=0= ===0 ···· =010 =011  341.00
10: Stockfish 4 [4c]      16.0/40 00== =0== 00== ==== ==== 0=== =0=0 ==1= =101 ···· =00=  321.00
11: RobboLito 0.21Q [4c]  15.0/40 0000 ==0= ==== ==0= 00== =0=1 0001 010= =100 =11= ····  292.75

220 of 440 games played
PGN of games is at :https://www.dropbox.com/s/praslvil9sqn7 ... al%207.pgn
enhorning
Posts: 342
Joined: Wed Jan 05, 2011 10:05 pm

Re: Survival of the Fittest 7

Post by enhorning »

75% done:

Code: Select all

    Engine                Score       Ho     St     Ko     Ko     Gu     Ho     St     Cr     St     Gu     Ro    S-B
01: Houdini 4 [4c]        38.0/60 ······ ===101 10==== ==1=== ===110 =0111= ==1=== 111=11 11==10 0=01=0 1111=1  1096.7
02: Stockfish 191113 [4c] 36.0/60 ===010 ······ ==10== 1===== =0=10= 11=111 0===1= 1110=1 =1==== 111=10 ==1===  1041.2
03: Komodo 5.1 [4c]       32.0/60 01==== ==01== ······ ===1=0 =00=1= ==01=0 =1=001 ==1=1= 11==01 =11=10 ======  941.50
04: Komodo 6 [4c]         31.5/60 ==0=== 0===== ===0=1 ······ =1=10= ===101 ==1=01 0==0== ====11 ===01= 11====  927.25
05: Gull 2.3 [4c]         31.0/60 ===001 =1=01= =11=0= =0=01= ······ 0111== 1====0 =0===1 ====0= =====1 ==1==0  926.00
06: Houdini 1.5a [4c]     30.0/60 =1000= 00=000 ==10=1 ===010 1000== ······ ====== =1=0=1 =1=1== 1=1=1= 11101=  854.75
07: Stockfish 3 JA [4c]   29.5/60 ==0=== 1===0= =0=110 ==0=10 0====1 ====== ······ 01===1 1===0= 1==0== =1=0==  880.00
08: Critter 1.6a [4c]     26.5/60 000=00 0001=0 ==0=0= 1==1== =1===0 =0=1=0 10===0 ······ ==0=1= ===1== 101==1  768.00
09: Stockfish 4 [4c]      26.0/60 00==01 =0==== 00==10 ====00 ====1= =0=0== 0===1= ==1=0= ······ =1011= =00===  779.50
10: Gull 2.2 [4c]         25.5/60 1=10=1 000=01 =00=01 ===10= =====0 0=0=0= 0==1== ===0== =0100= ······ =011==  777.75
11: RobboLito 0.21Q [4c]  24.0/60 0000=0 ==0=== ====== 00==== ==0==1 00010= =0=1== 010==0 =11=== =100== ······  715.25

330 of 440 games played
PGN : https://www.dropbox.com/s/praslvil9sqn7 ... al%207.pgn
enhorning
Posts: 342
Joined: Wed Jan 05, 2011 10:05 pm

Re: Survival of the Fittest 7

Post by enhorning »

Final standings:

Code: Select all

    Engine                Score         Ho       St       Ko       Ko       Gu       Ho       St       St       Cr       Gu       Ro    S-B
01: Houdini 4 [4c]        50.5/80 ········ ===101== 10====== ==1====0 ===1100= =0111=1= 11==10=1 ==1===11 111=1111 0=01=010 1111=1=1  1933.0
02: Stockfish 191113 [4c] 47.0/80 ===010== ········ ==10==0= 1======0 =0=10=0= 11=111== =1====== 0===1=11 1110=11= 111=10=1 ==1====1  1807.5
03: Komodo 5.1 [4c]       43.0/80 01====== ==01==1= ········ ===1=0=1 =00=1==0 ==01=0== 11==01== =1=001== ==1=1==0 =11=10=1 =======1  1691.0
04: Komodo 6 [4c]         43.0/80 ==0====1 0======1 ===0=1=0 ········ =1=10=0= ===101== ====111= ==1=01== 0==0==1= ===01=10 11====1=  1690.0
05: Gull 2.3 [4c]         42.5/80 ===0011= =1=01=1= =11=0==1 =0=01=1= ········ 0111==== ====0=== 1====0=1 =0===1=0 =====10= ==1==0==  1703.2
06: Houdini 1.5a [4c]     40.5/80 =1000=0= 00=000== ==10=1== ===010== 1000==== ········ =1=1==10 =======1 =1=0=1== 1=1=1==1 11101===  1549.5
07: Stockfish 4 [4c]      37.0/80 00==01=0 =0====== 00==10== ====000= ====1=== =0=0==01 ········ 0===1=11 ==1=0=1= =1011=10 =00===1=  1458.7
08: Stockfish 3 JA [4c]   37.0/80 ==0===00 1===0=00 =0=110== ==0=10== 0====1=0 =======0 1===0=00 ········ 01===111 1==0==1= =1=0====  1458.0
09: Critter 1.6a [4c]     34.5/80 000=0000 0001=00= ==0=0==1 1==1==0= =1===0=1 =0=1=0== ==0=1=0= 10===000 ········ ===1===1 101==1==  1343.2
10: Gull 2.2 [4c]         34.0/80 1=10=101 000=01=0 =00=01=0 ===10=01 =====01= 0=0=0==0 =0100=01 0==1==0= ===0===0 ········ =011==1=  1376.0
11: RobboLito 0.21Q [4c]  31.0/80 0000=0=0 ==0====0 =======0 00====0= ==0==1== 00010=== =11===0= =0=1==== 010==0== =100==0= ········  1240.7

440 games played / Tournament is finished
PGN with games is at : https://www.dropbox.com/s/praslvil9sqn7 ... al%207.pgn
Accumulated bayeselo from my survival tournaments:

Code: Select all

Rank Name                           Elo    +    - games score oppo. draws
   1 Houdini 4 [4c]                3130   45   45   120   65%  3026   44%
   2 Stockfish 191113 [4c]         3108   43   43   120   62%  3027   54%
   3 Komodo 6 [4c]                 3096   43   43   120   60%  3028   57%
   4 Komodo 5.1 [4c]               3068   43   43   120   55%  3031   58%
   5 Houdini 1.5a [4c]             3055   38   38   160   58%  2996   48%
   6 Gull 2.3 [4c]                 3051   43   43   120   53%  3032   60%
   7 Critter 1.6a [4c]             3032   38   38   160   55%  2998   52%
   8 Stockfish 3 JA [4c]           3031   38   38   160   54%  2998   54%
   9 Stockfish 4 [4c]              3030   43   43   120   49%  3034   60%
  10 Gull 2.2 [4c]                 3012   38   38   160   52%  2999   54%
  11 Bouquet 1.8 [4c]              2997   55   55    80   58%  2941   45%
  12 RobboLito 0.21Q [4c]          2978   38   38   160   46%  3002   54%
  13 Komodo CCT [1c]               2978   54   54    80   54%  2942   49%
  14 IvanHoe 9.46h [4c]            2949   54   54    80   51%  2944   46%
  15 Strelka 5.5 [1c]              2942   52   52    80   48%  2944   61%
  16 Komodo 5 [1c]                 2939   53   53    80   49%  2944   50%
  17 Hiarcs 14 [4c]                2919   53   53    80   46%  2945   54%
  18 Komodo 3 [1c]                 2909   46   46   120   56%  2858   39%
  19 Hannibal 1.4b [4c]            2904   56   56    80   44%  2946   39%
  20 Protector 1.5.0 [4c]          2901   46   46   120   55%  2859   40%
  21 Spike 1.4 [4c]                2831   57   57    80   55%  2793   33%
  22 BlackMamba 1.4 [1c]           2803   57   57    80   50%  2795   33%
  23 Rybka 2.3.2a [4c]             2798   46   46   120   41%  2864   44%
  24 Hiarcs 14 [1c]                2739   48   48   120   50%  2733   29%
  25 Spark 1.0 [4c]                2735   55   55    80   42%  2798   46%
  26 Quazar 0.4 [1c]               2698   42   42   160   62%  2601   28%
  27 Thinker 5.4d Inert [4c]       2694   57   57    80   36%  2800   35%
  28 Toga II 3.0 [4c]              2693   41   41   160   62%  2602   33%
  29 Crafty 23.6 [4c]              2682   37   37   200   52%  2663   36%
  30 Scorpio 2.7.6 [4c]            2668   41   41   160   59%  2603   35%
  31 MinkoChess 1.3 [4c]           2651   40   40   160   56%  2604   38%
  32 Doch 1.34 JA [1c]             2644   57   57    80   48%  2652   34%
  33 Booot 5.2.0 [1c]              2635   40   40   160   54%  2605   36%
  34 ExChess 7.18 [4c]             2627   41   41   160   53%  2605   29%
  35 Gaviota 0.86 [4c]             2610   41   41   160   51%  2606   29%
  36 Murka 3 [1c]                  2602   46   46   120   53%  2575   35%
  37 Octochess r5190 [4c]          2600   46   46   120   53%  2575   35%
  38 Tornado 4.88 [4c]             2598   57   57    80   51%  2591   33%
  39 Glaurung 2.2 [4c]             2595   56   56    80   51%  2591   31%
  40 Texel 1.02 [1c]               2592   47   47   120   52%  2576   28%
  41 Crafty 23.7 [4c]              2592   81   81    40   36%  2699   28%
  42 Gnuchess 5.50 [1c]            2587   83   83    40   48%  2609   20%
  43 Bright 0.5c [4c]              2582   79   79    40   34%  2700   38%
  44 Thinker 5.4d Passive [4c]     2575   80   80    40   46%  2610   33%
  45 Nemo 1.0.1 beta [1c]          2571   58   58    80   51%  2560   30%
  46 Loop 2007 [4c]                2569   80   80    40   33%  2700   35%
  47 Arasan 16.2 [4c]              2566   80   80    40   45%  2611   30%
  48 Ice 1.0 v1619 [1c]            2543   80   80    40   46%  2575   28%
  49 Twisted Logic 20100131x [1c]  2536   80   80    40   40%  2612   25%
  50 BugChess2 1.9 [4c]            2527   57   57    80   46%  2562   26%
  51 DiscoCheck 5.0.1 [1c]         2527   77   77    40   38%  2613   40%
  52 Arasan 16.1 [4c]              2520   77   77    40   45%  2549   35%
  53 Fruit 2.3.1 [1c]              2519   78   78    40   46%  2549   38%
  54 Bobcat 3.25 [4c]              2514   81   81    40   24%  2703   38%
  55 Shredder Classic 4 [1c]       2505   78   78    40   44%  2550   38%
  56 Cheng4 0.35a [4c]             2488   82   82    40   38%  2578   25%
  57 Dirty 20Apr2013 [1c]          2468   77   77    40   34%  2579   43%
  58 Rodent 1.1 [1c]               2425   83   83    40   30%  2581   20%
  59 Slow Chess Blitz WV2.1 [1c]   2403   84   84    40   30%  2555   25%
  60 Thinker 5.4d Active [4c]      2399   83   83    40   25%  2582   30%
  61 Deuterium 13.1.31.113 [1c]    2392   80   80    40   28%  2556   30%
  62 Ruffian 1.05 [1c]             2388   82   82    40   26%  2556   33%
  63 NirvanaChess 1.2 [1c]         2372   86   86    40   23%  2584   25%
  64 Pro Deo 1.85 [1c]             2359   89   89    40   26%  2557   18%
  65 Bison 9.11 [1c]               2351   92   92    40   16%  2621   18%
  66 Jonny 4 [4c]                  2263   98   98    40   15%  2562   15%
I'm planning on starting a new series soon, same format, though I will use a different opening book - I am considering the 2-move book there has been a couple of threads about.

These are my preliminary list of engines to include:
Houdini 4
Stockfish 191113
Komodo 6
Komodo 5.1
Houdini 1.5a
Gull 2.3
Critter 1.6a
Stockfish 3 JA
Stockfish 4
Stockfish DD
Gull 2.2
Bouquet 1.8
Robbolito 0.21Q
Komodo CCT
IvanHoe 9.46h
Strelka 5.5
Komodo 5
Komodo 3
Hiarcs 14
Protector 1.5.0
Hannibal 1.4b
Komodo 4
Komodo 2.03
Spike 1.4
Komodo 2.01
BlackMamba 1.4
Rybka 2.3.2a
Junior Yokohama
Junior 13
Junior 13.3
Hiarcs 14
Spark 1.0
Komodo 1.3
Quazar 0.4
Thinker 5.4d Inert
Toga II 3.0
Crafty 23.6
Scorpio 2.7.6 JA
MinkoChess 1.3
Doch 1.3.4 JA
Booot 5.2.0
ExChess 7.18
Crafty 23.8
Arasan 16.3
Bright 0.5c
Gaviota 0.86
Glaurung 2.2
Crafty 23.7
Tornado 4.88
Murka 3
OctoChess r5190
Thinker 5.4d Passive
Texel 1.02
Arasan 16.2
SmarThink 1.40
Loop 2007 (13.6)
Cheng4 0.36a
Nemo 1.0.1 beta
Gnu Chess 5.50
DiscoCheck 5.1
DiscoCheck 5.0.1
Arasan 16.1
BugChess2 1.9
Bobcat 3.25
Twisted Logic 20100131x
Fruit 2.3.1
Shredder Classic 4
Gnu Chess 6.1.1
Cheng4 0.35a
Thinker 5.4d Active
RedQueen 1.1.4
Ice 1.0 v1619
Naraku 1.4
Frenzee 3.5.19
Dirty 20Apr2013
Chronos 1.9.9
NirvanaChess 1.3
Alfil 13.1
Delfi 5.4
Rodent 1.2
Naum 2.0
Atlas 3.60
Jonny 4
Pharaon 3.5.1
Bison 9.11
Daydreamer 1.75
Deuterium 13.1.31.113
Rodent 1.1
Hamsters 0.7.1
Alaric 707
Zappa 1.1
Brutus 8.05
ProDeo 1.85
Cyrano 0.6b17
Pawny 1.0
Slow Chess Blitz WV2.1
E.T. Chess 13.01.08
Colossus 2008b
Nebula 2.0
Philou 3.7.1
WildCat 8
Movei 00.8.438 (10 10 10)
Godel 2.3.7
NirvanaChess 1.2
Djinn 0.992
Vajolet 2.48
Ruffian 1.0.5
Sloppy 0.2.2
Rotor 0.8
Tucano 3.00
List 5.12
Glass 2.0
Hermann 2.8
GarboChess 3.0
Arminius 2013-08-10
... and if I end up purchasing them:
Komodo TCEC
Rybka 4.1
Vitruvius 1.11C
Chiron 1.5

Suggestions for other engines to include are welcome, and new releases while the series is running will of course be considered as well.