Chess.com 2018 computer chess championship

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: Chess.com 2018 computer chess championship

Post by Javier Ros » Wed Sep 26, 2018 3:44 pm

chrisw wrote:
Wed Sep 26, 2018 9:32 am
Guenther wrote:
Wed Sep 26, 2018 6:14 am
This is the table after some cleaning up (not only the unterminated, but also the wrong game headers)
Game 1 which was still testing and was played with no opening moves at all still counts...
Andscacs instead of 1. e4 c5 2. Nf3 h6 3. c3 vs. SF played from the start position and the testing game
was elevated to a real stage 2 game later.

The normally reversed opening pair for that game:
[Event "TESTING for Stage 2"]
[Site "Chess.com"]
[Date "2018.09.17"]
[Round "?"]
[White "Andscacs 0.94"]
[Black "Stockfish 220818"]
[Result "0-1"]
[WhiteElo "3244"]
[BlackElo "3300"]
[ECO "C01"]
[TimeControl "900+5"]
[Time "10:23:16"]
[Termination "normal"]
[PlyCount "118"]

1. e4 {+0.20/26 21}
1... e6 {-0.15/33 50}
2. d4 {+0.29/29 29}
2... d5 {-0.11/32 8}
3. exd5 {+0.27/29 67}

[Event "CCCC 1: Rapid Rumble Stage 2"]
[Site "Chess.com"]
[Date "2018.09.18"]
[Round "?"]
[White "Stockfish 220818"]
[Black "Andscacs 0.94"]
[Result "1-0"]
[WhiteElo "3439"]
[BlackElo "3300"]
[ECO "B27"]
[TimeControl "900+5"]
[Time "08:39:57"]
[Termination "normal"]
[PlyCount "147"]

1. e4 c5 2. Nf3 h6 3. c3 {+0.77/32 50}

Code: Select all

CCCC 1: Rapid Rumble (15|5) Stage 2
Chess.com, 2018.09.17 - 2018.09.25
Average Rating: 3430  (Category 48)
                     Rtng    Score              1          2          3          4          5          6          7          8   Perf Chg
------------------------------------------------------------------------------------------------------------------------------------------
 1: Stockfish 220818 3439  51.0 / 70   XXXXXXXXXX =1=1=1=1=1 ===1=====1 ====1===== 1=1=1=1=11 11=11=1=1= =1=111===1 11=11111==   3596 +140  (+32 -0 =38)
 2: Houdini 6.03     3439  42.0 / 69   =0=0=0=0=0 XXXXXXXXXX 1==1==01=. ========1= 1==1=1==== 11=11=1==1 =1===1=1=1 1=1=1===1=   3501 +55  (+21 -6 =42)
 3: Komodo 2118.00   3439  38.5 / 69   ===0=====0 0==0==10=. XXXXXXXXXX ======0=01 1====1===1 1==101==== 1==1==1==1 1==1=1=1==   3465 +21  (+16 -8 =45)
 4: Lc0 17.11089     3439  37.5 / 70   ====0===== ========0= ======1=10 XXXXXXXXXX ========10 =111=0==== 11====1=== 10====101=   3451  +7  (+12 -7 =51)
 5: Ethereal 10.88   3404  31.0 / 70   0=0=0=0=00 0==0=0==== 0====0===0 ========01 XXXXXXXXXX ==0=0====1 ======1=== 1=1=10=11=   3386 -14  (+8 -16 =46)
 6: Fire 7.1         3404  29.0 / 70   00=00=0=0= 00=00=0==0 0==010==== =000=1==== ==1=1====0 XXXXXXXXXX ===1=1=1== =====0=1==   3364 -35  (+8 -20 =42)
 7: Booot 6.3.1      3439  25.5 / 70   =0=000===0 =0===0=0=0 0==0==0==0 00====0=== ======0=== ===0=0=0== XXXXXXXXXX ====1=====   3321 -119  (+1 -20 =49)
 8: Andscacs 0.94    3439  24.5 / 70   00=00000== 0=0=0===0= 0==0=0=0== 01====010= 0=0=01=00= =====1=0== ====0===== XXXXXXXXXX   3310 -126  (+4 -25 =41)
------------------------------------------------------------------------------------------------------------------------------------------
279 games: +79 -23 =177
I want to suggest the elo of LC0 somehow manages to match the elo of whatever it is playing against. Well, that's one "explanation" of its rather curious behaviour:

If one looks at the loss count of the second program, Houdini against the final ranking of its opponents, we get, as would be expected, an decreasing gradient: 5,1,0,0,0,0,0
Komodo gets: 2,3,2,0,1,0,0
Ethereal: 6,3,3,1,2,0,1
Fire: 6,6,3,3,1,0,1
Booot: 5,4,4,3,1,3,0
Andsacs: 7,4,4,3,5,1,1

Lc0 is different: 1,1,1,1,1,0,2, almost irrelevent who the opponent is, the loss rate remains almost constant.

Obviously, "non-losses", counting wins and draws together, shows the same pattern in reverse. Which suggests, well, to me, that LC0 doesn't really have an elo that can be mapped onto any particular opponent. It's not behaving itself properly according to the laws of elo ratings.
This constant loss rate is mainly due to the tactical weakness of lc0 that, when it appears against any alpha-beta program always leads to defeat.
On the other hand, this tactical weakness is practically irrelevant against humans, unable to detect it and take advantage of it in most cases.

So, perhaps we should say that lc0 is not behaving itself according to the laws of elo of the classical alpha-beta chess programs.
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

jdart
Posts: 3826
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Chess.com 2018 computer chess championship

Post by jdart » Thu Sep 27, 2018 11:07 pm

Are games from the completed parts of the tourney available somewhere?

--Jon

User avatar
Guenther
Posts: 3067
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Fri Sep 28, 2018 6:38 am

jdart wrote:
Thu Sep 27, 2018 11:07 pm
Are games from the completed parts of the tourney available somewhere?

--Jon
Official CCCC archives stage 1+2
https://cccfiles.chess.com/archive/ccc1 ... /games.pgn
https://cccfiles.chess.com/archive/ccc1 ... /games.pgn

Inofficial current stage 3 (final) games
http://newman.chess.com/games.pgn

Note that stage 1 and stage 2 are not cleaned up files and adjudication results are missing for the
unterminated games. Also one game is missing per file.
Current foe list count : [99]
http://rwbc-chess.de/chronology.htm

User avatar
Guenther
Posts: 3067
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Fri Sep 28, 2018 6:51 am

Guenther wrote:
Wed Sep 26, 2018 8:28 am

Code: Select all

   # PLAYER              :   RATING   ERROR  POINTS  PLAYED   (%)
   1 Stockfish 220818    :  3663.79   52.89    90.0     116  77.6
   2 Houdini 6.03        :  3584.63   41.68    79.0     115  68.7
   3 Komodo 2118.00      :  3565.40   46.24    76.5     115  66.5
   4 Lc0 17.11089        :  3513.30   47.34    68.5     115  59.6
   5 Fire 7.1            :  3465.80   43.57    62.0     116  53.4
   6 Ethereal 10.88      :  3459.26   41.68    61.0     116  52.6
   7 Booot 6.3.1         :  3416.64   44.68    54.5     116  47.0
   8 Andscacs 0.94       :  3400.08   44.39    52.0     116  44.8
   9 Shredder 13         :  3370.30   62.61    26.5      46  57.6
  10 Xiphos 0.3.17       :  3324.13   63.92    24.0      46  52.2
  11 Fritz 16.10         :  3324.13   60.07    24.0      46  52.2
  12 Texel 1.07          :  3305.80   63.57    23.0      46  50.0
  13 Laser 1.6           :  3289.81   60.37    21.0      44  47.7
  14 Pedone 1.8          :  3269.06   64.72    21.0      46  45.7
  15 Vajolet 2.6         :  3269.06   62.75    21.0      46  45.7
  16 Arasan CCCC-2018    :  3250.55   61.77    20.0      46  43.5
  17 Fizbo 1.9           :  3241.23   68.62    19.5      46  42.4
  18 Gull 3.syz          :  3222.42   70.70    18.5      46  40.2
  19 Wasp 3.25           :  3164.08   72.15    15.5      46  33.7
  20 Nemorino 5.00       :  3143.72   70.10    14.5      46  31.5
  21 Ivanhoe 999946h     :  3112.00   72.30    13.0      46  28.3
  22 Senpai 2.0          :  3042.52   80.28    10.0      46  21.7
  23 Nirvana 2.4         :  2989.46   90.39     8.0      46  17.4
  24 Crafty 25.2         :  2932.85  104.85     6.0      45  13.3

White advantage = 61.91 +/- 8.17
Draw rate (equal opponents) = 72.56 % +/- 2.60

(based on average rating of 3305 see above)
Actualised after 72 games in the final, with the same calculation method and assumptions:

Code: Select all

   # PLAYER              :   RATING   ERROR  POINTS  PLAYED   (%)
   1 Stockfish 220818    :  3657.20   41.36   132.0     188  70.2
   2 Houdini 6.03        :  3589.18   40.50   109.0     187  58.3
   3 Komodo 2118.00      :  3564.86   42.28    76.5     115  66.5
   4 Lc0 17.11089        :  3512.94   45.25    68.5     115  59.6
   5 Fire 7.1            :  3465.54   37.90    62.0     116  53.4
   6 Ethereal 10.88      :  3459.02   42.84    61.0     116  52.6
   7 Booot 6.3.1         :  3416.49   40.92    54.5     116  47.0
   8 Andscacs 0.94       :  3399.97   40.10    52.0     116  44.8
   9 Shredder 13         :  3370.24   62.11    26.5      46  57.6
  10 Xiphos 0.3.17       :  3324.15   63.11    24.0      46  52.2
  11 Fritz 16.10         :  3324.15   62.58    24.0      46  52.2
  12 Texel 1.07          :  3305.86   65.03    23.0      46  50.0
  13 Laser 1.6           :  3289.88   59.45    21.0      44  47.7
  14 Vajolet 2.6         :  3269.19   62.39    21.0      46  45.7
  15 Pedone 1.8          :  3269.19   60.36    21.0      46  45.7
  16 Arasan CCCC-2018    :  3250.71   65.01    20.0      46  43.5
  17 Fizbo 1.9           :  3241.40   63.69    19.5      46  42.4
  18 Gull 3.syz          :  3222.63   64.67    18.5      46  40.2
  19 Wasp 3.25           :  3164.39   67.80    15.5      46  33.7
  20 Nemorino 5.00       :  3144.07   65.87    14.5      46  31.5
  21 Ivanhoe 999946h     :  3112.39   77.37    13.0      46  28.3
  22 Senpai 2.0          :  3043.02   74.59    10.0      46  21.7
  23 Nirvana 2.4         :  2990.04   83.16     8.0      46  17.4
  24 Crafty 25.2         :  2933.47  102.07     6.0      45  13.3

White advantage = 59.74 +/- 7.87
Draw rate (equal opponents) = 74.87 % +/- 2.32
Current foe list count : [99]
http://rwbc-chess.de/chronology.htm

corres
Posts: 1683
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Chess.com 2018 computer chess championship

Post by corres » Fri Sep 28, 2018 7:56 am

Robert Pope wrote:
Wed Sep 26, 2018 3:15 pm
I think the issue is that LC0 plays very well, except for a glaring hole that almost any opponent can capitalize on. Other engines have their own holes, but they are subtle, so weaker engines are less likely to discover them.
So we can say LC0 capitalize the weaker positional knowledge of AB engines and AB engines capitalize the weaker tactical knowledge of AB engines.
Where is the engine with positional knowledge of LC0 and tactical knowledge of AB engines?

yanquis1972
Posts: 1762
Joined: Tue Jun 02, 2009 10:14 pm

Re: Chess.com 2018 computer chess championship

Post by yanquis1972 » Fri Sep 28, 2018 8:03 am

Not far off. Will continue to be shocked there isn’t an adequate if not necessarily legal jerryrigged hybrid floating around. If lc0 + 3rd party doesn’t do it my money would be on Komodo or Houdini making a splash commercially.

ernest
Posts: 1860
Joined: Wed Mar 08, 2006 7:30 pm

Re: Chess.com 2018 computer chess championship

Post by ernest » Thu Oct 04, 2018 5:41 pm

What was the final result of the CCCC final : 120-80, 119-81 ?
Could find it nowhere...

Paloma
Posts: 849
Joined: Thu Dec 25, 2008 8:07 pm

Re: Chess.com 2018 computer chess championship

Post by Paloma » Thu Oct 04, 2018 7:10 pm

ernest ,

i found this Link: https://www.chess.com/news/view/stockfi ... s-3rd-3887
120 : 80

ernest
Posts: 1860
Joined: Wed Mar 08, 2006 7:30 pm

Re: Chess.com 2018 computer chess championship

Post by ernest » Thu Oct 04, 2018 8:22 pm

Paloma wrote:
Thu Oct 04, 2018 7:10 pm
ernest ,

i found this Link: https://www.chess.com/news/view/stockfi ... s-3rd-3887
120 : 80
Thanks ! How did you find that link ?
I had an older link, which never got updated...

jp
Posts: 825
Joined: Mon Apr 23, 2018 5:54 am

Re: Chess.com 2018 computer chess championship

Post by jp » Mon Oct 08, 2018 6:42 am

On chess.com: "CCC2 is on hold until we can figure out the Lc0 timeout issue, and why some engines are going to 1 thread."

What is the Lc0 timeout issue??

Post Reply