CCRL 40/40, 40/4 and FRC lists updated (28th October 2017)

[Adam Hair]

I'm surprised to see a drop of more than 40 Elo from CT800 1.11 to 1.12, that didn't show up during testing. Some questions:

from CCRL 40/4 full list:

Code: Select all

CT800 1.11 64-bit       2240    +26     -26     45.8%   +32.7   19.7%   532     54.7%
CT800 1.12 64-bit       2194    +33     -33     43.2%   +49.8   19.7%   330     49.4%
NG-Play 9.86 64-bit     2187    +18     -18     50.5%   -3.1    22.2%   1079    53.5%

The error bars look still very high? Did you expect version 1.11 that much
above NG-Play? May be 1.11 was just a bit lucky so far and 1.12 will rise.

Of course a check of the games won't hurt.

Guenther

Edit: It seems the good result of 1.11 is primarily based on an extreme outsider result vs. Devel? (which produced strange outsider results so far en masse, if I see it right)

From my own testing, v1.12 seems to be a little weaker than v1.11 (40/140" tc, 2 move starting positions taken from GM games and filtered with Komodo):

Code: Select all

   # PLAYER                        :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)
   1 TJchess 1.3 64-bit            :  2397.0   ----    47.5      64    74     100
   2 Jumbo 0.5.3 64-bit            :  2341.0   ----    41.5      64    65     100
   3 Barbarossa 0.4.0 64-bit       :  2332.0   ----    39.0      64    61     100
   4 Sage 3.53                     :  2326.0   ----    35.5      65    55     100
   5 Ayito 0.2.994                 :  2288.0   ----    31.5      64    49     100
   6 Paladin 0.1 64-bit            :  2273.0   ----    39.5      64    62     100
   7 Esc 1.16                      :  2267.0   ----    24.5      64    38     100
   8 Zeus 1.29                     :  2258.0   ----    39.5      64    62      50
   9 CT800 1.11 64-bit             :  2257.8   32.5   218.0     443    49      90
  10 Ant 2006-F                    :  2237.0   ----    28.5      63    45     100
  11 Adam 3.3                      :  2226.0   ----    24.5      58    42      52
  12 CT800 1.12 64-bit             :  2225.2   29.8   198.5     443    45      94
  13 RattateChess 1.0 Nosferatu    :  2201.0   ----    25.5      65    39     100
  14 Latista (Sinobyl) 1.5         :  2193.0   ----    25.5      62    41     100
  15 Dorpsgek Dillinger 64-bit     :  2189.0   ----    35.5      61    58     100
  16 Baislicka 1.0 64-bit          :  2149.0   ----    31.5      64    49     ---

White advantage = 9.27 +/- 10.29
Draw rate (equal opponents) = 20.05 % +/- 1.23

[Ras]

From my own testing, v1.12 seems to be a little weaker than v1.11

I think I have found the reason: the time management is now much closer to the embedded version. In games with fixed time per game, the target is being down to 50% by move 30 and 35% by move 40. Of course, that leaves less time for the late game phase.

The PC version of V1.11 had the time management from NG-Play, which assumes that there are always 40 moves left to play in "time per game" mode.

The reason is that the engine is targeted against humans so that more time in the early phase is necessary to reach better depth despite the relatively weak hardware.