NEBB-Rankinglists: Critter 1.4

[pohl4711]

The NEBB-Rankingslists (Naked Engine Bullet and Blitz) now with Critter 1.4:

Intel Q9550 2.83GHz Quad (no SSE support, Vista 64bit), LittleBlitzerGUI, 256 MB Hash, 1 Core per Engine, no ponder, no bases, no resign. 50 super-short test-positions (1.a3 a6, 1.a3 b6, 1.a3 c6…..1.h3 g6, 1.h3 h6) = Naked Engines (no openings (book or long test-positions (Noomen etc.)), no endgame-databases) – only engine-thinking from move 2 until mate or draw.
Two lists with exact same conditions except the thinking time. That makes it possible to see, which engine scores better or worse with more or less thinking time...

Blitzlist (4’+2’’)

Code: Select all

Rank Name                       Elo    +    - games score oppo. draws 
   1 Houdini 2.0c x64          3104   18   18   800   60%  3041   40% 
   2 Houdini 1.5a x64          3100   18   18   800   59%  3041   41% (best freeware) 
   3 Critter 1.4 64-bit        3083   19   19   700   55%  3052   47% 
   4 Komodo 4 x64              3074   18   18   800   53%  3053   43% (singlecore) 
   5 Critter 1.2 64-bit        3055   18   18   800   51%  3049   45% 
   6 Ivanhoe B46fa x64         3043   16   16   900   48%  3054   51% 
   7 Komodo 3 x64              3031   19   19   700   47%  3049   46% (singlecore) 
   8 Rybka 4.1 x64             3028   17   17   900   46%  3056   46% 
   9 RobboLito 0.09 x64        3013   17   17   900   43%  3058   50% (singlecore) 
  10 Stockfish 2.1.1 JA 64bit  3000   17   17   900   41%  3059   42%

Bulletlist (1’+500 ms)

Code: Select all

Rank Name                       Elo    +    - games score oppo. draws 
   1 Houdini 2.0c x64          3122   18   18   800   63%  3039   37% 
   2 Houdini 1.5a x64          3102   18   18   800   60%  3039   38% (best freeware) 
   3 Critter 1.4 64-bit        3094   19   19   700   57%  3050   43% 
   4 Critter 1.2 64-bit        3067   18   18   800   53%  3046   43% 
   5 Komodo 4 x64              3054   18   18   800   50%  3057   38% (singlecore) 
   6 Ivanhoe B46fa x64         3046   17   17   900   49%  3054   47% 
   7 Komodo 3 x64              3023   19   19   700   46%  3052   37% (singlecore) 
   8 Rybka 4.1 x64             3019   17   17   900   44%  3057   40% 
   9 RobboLito 0.09 x64        3007   17   17   900   42%  3058   44% (singlecore) 
  10 Stockfish 2.1.1 JA 64bit  3000   17   17   900   41%  3059   38%

The new Stockfish 2.2 is buggy and loses 40% of all games in the bulletlist-testrun on time!!! So I have to wait for a bugfix before testing Stockfish 2.2.

Greetings – Stefan

[Houdini]

Two lists with exact same conditions except the thinking time. That makes it possible to see, which engine scores better or worse with more or less thinking time...

Unless you play more games, there is really nothing that can be concluded by comparing the two lists.
To compare the progression of two engines when using more thinking time, we need 4 individual ratings. The combined error bar on the comparison will be twice the individual error bar or +- 35 Elo.

Robert