Chess.com 2018 computer chess championship

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
chrisw
Posts: 2013
Joined: Tue Apr 03, 2012 2:28 pm

Re: Chess.com 2018 computer chess championship

Post by chrisw » Sat Sep 22, 2018 4:54 pm

Laskos wrote:
Sat Sep 22, 2018 7:35 am
Uri Blass wrote:
Sat Sep 22, 2018 6:48 am
Nay Lin Tun wrote:
Sat Sep 22, 2018 1:44 am
Updating status!

Leela's chance of going into final is thinner and thinner!

Anyway, 11250 + TB should be +50 elo than current network and she will have high chance of going into final in October 5/2 blitz tourney.
I can only hope it is not going to happen because I do not like stupid engines to get good results.
LOL, that's brilliant!

Exactly for this I like Lc0. Maybe because I empathize with it, as I am stupid too, but I want to get good results :D.
In fact the best thing for me would be "Idiot's guide on how to have a shot at Nobel Prize being a complete idiot" :lol:.

lc0 evaluated a drawn tablebases KQ vs KQP as 15 pawns for herself.
Lack of tablebases is no excuse because normal engines without them are not going to do it.
Yeah, I'm still trying to work out whether to "like" LC0 or not. It would be easy if it (she?) was in a vacuum, but there's a lot of baggage associated.

Nay Lin Tun
Posts: 514
Joined: Mon Jan 16, 2012 5:34 am

Re: Chess.com 2018 computer chess championship

Post by Nay Lin Tun » Sun Sep 23, 2018 11:19 am

Leela's first win against Komodo! ( opening book - 2 moves)



Leela got 3rd position after this game.
Her performance would be second best if there were only top 4 engines!
Attachments
Capture.PNG
Capture.PNG (14.68 KiB) Viewed 2441 times

chrisw
Posts: 2013
Joined: Tue Apr 03, 2012 2:28 pm

Re: Chess.com 2018 computer chess championship

Post by chrisw » Sun Sep 23, 2018 12:53 pm

Nay Lin Tun wrote:
Sun Sep 23, 2018 11:19 am
Leela's first win against Komodo! ( opening book - 2 moves)



Leela got 3rd position after this game.
Her performance would be second best if there were only top 4 engines!
This was lost by Komodo, not won by Leela. On move 3 Komodo gave away a pawn, and that's the way it stayed. Pieces were exchanged, any pressure Komodo had evaporated, but Leela had a bad bishop. Which Leela then proceeded to make worse by putting all her pawns on the same colour squares. Then the interminable shuffling, with Leela presumably misevaluating and unable to work out that she had to advance a pawn onto a white square somehow. The reason she eventually did, had nothing to do with playing chess well, but only in order to avoid the 50 move draw rule. And as a result of Leela being forced to do what she didn't want to do, her position improved and Komodo went downhill.
So, actually, Komodo played not a bad defensive game at all, and Leela blundered into a win, effectively. Nothing at all to crow about, I'm afraid.

nabildanial
Posts: 104
Joined: Thu Jun 05, 2014 3:29 am
Location: Malaysia

Re: Chess.com 2018 computer chess championship

Post by nabildanial » Tue Sep 25, 2018 8:04 am

Leela wins again vs Komodo.



User avatar
Guenther
Posts: 3015
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Wed Sep 26, 2018 5:54 am

A pity that it is again incomplete like stage 1 already.
http://talkchess.com/forum3/viewtopic.p ... 75#p774375
Current foe list count : [93]
http://rwbc-chess.de/chronology.htm

User avatar
Guenther
Posts: 3015
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Wed Sep 26, 2018 6:14 am

This is the table after some cleaning up (not only the unterminated, but also the wrong game headers)
Game 1 which was still testing and was played with no opening moves at all still counts...
Andscacs instead of 1. e4 c5 2. Nf3 h6 3. c3 vs. SF played from the start position and the testing game
was elevated to a real stage 2 game later.

The normally reversed opening pair for that game:
[Event "TESTING for Stage 2"]
[Site "Chess.com"]
[Date "2018.09.17"]
[Round "?"]
[White "Andscacs 0.94"]
[Black "Stockfish 220818"]
[Result "0-1"]
[WhiteElo "3244"]
[BlackElo "3300"]
[ECO "C01"]
[TimeControl "900+5"]
[Time "10:23:16"]
[Termination "normal"]
[PlyCount "118"]

1. e4 {+0.20/26 21}
1... e6 {-0.15/33 50}
2. d4 {+0.29/29 29}
2... d5 {-0.11/32 8}
3. exd5 {+0.27/29 67}

[Event "CCCC 1: Rapid Rumble Stage 2"]
[Site "Chess.com"]
[Date "2018.09.18"]
[Round "?"]
[White "Stockfish 220818"]
[Black "Andscacs 0.94"]
[Result "1-0"]
[WhiteElo "3439"]
[BlackElo "3300"]
[ECO "B27"]
[TimeControl "900+5"]
[Time "08:39:57"]
[Termination "normal"]
[PlyCount "147"]

1. e4 c5 2. Nf3 h6 3. c3 {+0.77/32 50}

Code: Select all

CCCC 1: Rapid Rumble (15|5) Stage 2
Chess.com, 2018.09.17 - 2018.09.25
Average Rating: 3430  (Category 48)
                     Rtng    Score              1          2          3          4          5          6          7          8   Perf Chg
------------------------------------------------------------------------------------------------------------------------------------------
 1: Stockfish 220818 3439  51.0 / 70   XXXXXXXXXX =1=1=1=1=1 ===1=====1 ====1===== 1=1=1=1=11 11=11=1=1= =1=111===1 11=11111==   3596 +140  (+32 -0 =38)
 2: Houdini 6.03     3439  42.0 / 69   =0=0=0=0=0 XXXXXXXXXX 1==1==01=. ========1= 1==1=1==== 11=11=1==1 =1===1=1=1 1=1=1===1=   3501 +55  (+21 -6 =42)
 3: Komodo 2118.00   3439  38.5 / 69   ===0=====0 0==0==10=. XXXXXXXXXX ======0=01 1====1===1 1==101==== 1==1==1==1 1==1=1=1==   3465 +21  (+16 -8 =45)
 4: Lc0 17.11089     3439  37.5 / 70   ====0===== ========0= ======1=10 XXXXXXXXXX ========10 =111=0==== 11====1=== 10====101=   3451  +7  (+12 -7 =51)
 5: Ethereal 10.88   3404  31.0 / 70   0=0=0=0=00 0==0=0==== 0====0===0 ========01 XXXXXXXXXX ==0=0====1 ======1=== 1=1=10=11=   3386 -14  (+8 -16 =46)
 6: Fire 7.1         3404  29.0 / 70   00=00=0=0= 00=00=0==0 0==010==== =000=1==== ==1=1====0 XXXXXXXXXX ===1=1=1== =====0=1==   3364 -35  (+8 -20 =42)
 7: Booot 6.3.1      3439  25.5 / 70   =0=000===0 =0===0=0=0 0==0==0==0 00====0=== ======0=== ===0=0=0== XXXXXXXXXX ====1=====   3321 -119  (+1 -20 =49)
 8: Andscacs 0.94    3439  24.5 / 70   00=00000== 0=0=0===0= 0==0=0=0== 01====010= 0=0=01=00= =====1=0== ====0===== XXXXXXXXXX   3310 -126  (+4 -25 =41)
------------------------------------------------------------------------------------------------------------------------------------------
279 games: +79 -23 =177
Current foe list count : [93]
http://rwbc-chess.de/chronology.htm

User avatar
Guenther
Posts: 3015
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Chess.com 2018 computer chess championship

Post by Guenther » Wed Sep 26, 2018 8:28 am

Ordo calculation of ratings after stage 1 + 2 combined (3 missing games) with 300 simuls.
Based on CCRL 40/40 average rating of Texel + Laser (4CPU) plus 3.5*35 rating points.
Texel and Laser placed in the middle range, therefore selected.
(48 physical cores = 3.5 doublings to 4 cores, 35 per doubling cautiously assumed between 4 and 48, YMMV)

Code: Select all


CCRL 40/40 Complete rating list

Laser 1.6 64-bit 4CPU	3223	+23	−23	40.8%
Texel 1.07 64-bit 4CPU	3143	+15	−15	49.8%

3223+3143/2=3183
3183+(3.5*35)~3305

Code: Select all

   # PLAYER              :   RATING   ERROR  POINTS  PLAYED   (%)
   1 Stockfish 220818    :  3663.79   52.89    90.0     116  77.6
   2 Houdini 6.03        :  3584.63   41.68    79.0     115  68.7
   3 Komodo 2118.00      :  3565.40   46.24    76.5     115  66.5
   4 Lc0 17.11089        :  3513.30   47.34    68.5     115  59.6
   5 Fire 7.1            :  3465.80   43.57    62.0     116  53.4
   6 Ethereal 10.88      :  3459.26   41.68    61.0     116  52.6
   7 Booot 6.3.1         :  3416.64   44.68    54.5     116  47.0
   8 Andscacs 0.94       :  3400.08   44.39    52.0     116  44.8
   9 Shredder 13         :  3370.30   62.61    26.5      46  57.6
  10 Xiphos 0.3.17       :  3324.13   63.92    24.0      46  52.2
  11 Fritz 16.10         :  3324.13   60.07    24.0      46  52.2
  12 Texel 1.07          :  3305.80   63.57    23.0      46  50.0
  13 Laser 1.6           :  3289.81   60.37    21.0      44  47.7
  14 Pedone 1.8          :  3269.06   64.72    21.0      46  45.7
  15 Vajolet 2.6         :  3269.06   62.75    21.0      46  45.7
  16 Arasan CCCC-2018    :  3250.55   61.77    20.0      46  43.5
  17 Fizbo 1.9           :  3241.23   68.62    19.5      46  42.4
  18 Gull 3.syz          :  3222.42   70.70    18.5      46  40.2
  19 Wasp 3.25           :  3164.08   72.15    15.5      46  33.7
  20 Nemorino 5.00       :  3143.72   70.10    14.5      46  31.5
  21 Ivanhoe 999946h     :  3112.00   72.30    13.0      46  28.3
  22 Senpai 2.0          :  3042.52   80.28    10.0      46  21.7
  23 Nirvana 2.4         :  2989.46   90.39     8.0      46  17.4
  24 Crafty 25.2         :  2932.85  104.85     6.0      45  13.3

White advantage = 61.91 +/- 8.17
Draw rate (equal opponents) = 72.56 % +/- 2.60

(based on average rating of 3305 see above)
Current foe list count : [93]
http://rwbc-chess.de/chronology.htm

chrisw
Posts: 2013
Joined: Tue Apr 03, 2012 2:28 pm

Re: Chess.com 2018 computer chess championship

Post by chrisw » Wed Sep 26, 2018 9:32 am

Guenther wrote:
Wed Sep 26, 2018 6:14 am
This is the table after some cleaning up (not only the unterminated, but also the wrong game headers)
Game 1 which was still testing and was played with no opening moves at all still counts...
Andscacs instead of 1. e4 c5 2. Nf3 h6 3. c3 vs. SF played from the start position and the testing game
was elevated to a real stage 2 game later.

The normally reversed opening pair for that game:
[Event "TESTING for Stage 2"]
[Site "Chess.com"]
[Date "2018.09.17"]
[Round "?"]
[White "Andscacs 0.94"]
[Black "Stockfish 220818"]
[Result "0-1"]
[WhiteElo "3244"]
[BlackElo "3300"]
[ECO "C01"]
[TimeControl "900+5"]
[Time "10:23:16"]
[Termination "normal"]
[PlyCount "118"]

1. e4 {+0.20/26 21}
1... e6 {-0.15/33 50}
2. d4 {+0.29/29 29}
2... d5 {-0.11/32 8}
3. exd5 {+0.27/29 67}

[Event "CCCC 1: Rapid Rumble Stage 2"]
[Site "Chess.com"]
[Date "2018.09.18"]
[Round "?"]
[White "Stockfish 220818"]
[Black "Andscacs 0.94"]
[Result "1-0"]
[WhiteElo "3439"]
[BlackElo "3300"]
[ECO "B27"]
[TimeControl "900+5"]
[Time "08:39:57"]
[Termination "normal"]
[PlyCount "147"]

1. e4 c5 2. Nf3 h6 3. c3 {+0.77/32 50}

Code: Select all

CCCC 1: Rapid Rumble (15|5) Stage 2
Chess.com, 2018.09.17 - 2018.09.25
Average Rating: 3430  (Category 48)
                     Rtng    Score              1          2          3          4          5          6          7          8   Perf Chg
------------------------------------------------------------------------------------------------------------------------------------------
 1: Stockfish 220818 3439  51.0 / 70   XXXXXXXXXX =1=1=1=1=1 ===1=====1 ====1===== 1=1=1=1=11 11=11=1=1= =1=111===1 11=11111==   3596 +140  (+32 -0 =38)
 2: Houdini 6.03     3439  42.0 / 69   =0=0=0=0=0 XXXXXXXXXX 1==1==01=. ========1= 1==1=1==== 11=11=1==1 =1===1=1=1 1=1=1===1=   3501 +55  (+21 -6 =42)
 3: Komodo 2118.00   3439  38.5 / 69   ===0=====0 0==0==10=. XXXXXXXXXX ======0=01 1====1===1 1==101==== 1==1==1==1 1==1=1=1==   3465 +21  (+16 -8 =45)
 4: Lc0 17.11089     3439  37.5 / 70   ====0===== ========0= ======1=10 XXXXXXXXXX ========10 =111=0==== 11====1=== 10====101=   3451  +7  (+12 -7 =51)
 5: Ethereal 10.88   3404  31.0 / 70   0=0=0=0=00 0==0=0==== 0====0===0 ========01 XXXXXXXXXX ==0=0====1 ======1=== 1=1=10=11=   3386 -14  (+8 -16 =46)
 6: Fire 7.1         3404  29.0 / 70   00=00=0=0= 00=00=0==0 0==010==== =000=1==== ==1=1====0 XXXXXXXXXX ===1=1=1== =====0=1==   3364 -35  (+8 -20 =42)
 7: Booot 6.3.1      3439  25.5 / 70   =0=000===0 =0===0=0=0 0==0==0==0 00====0=== ======0=== ===0=0=0== XXXXXXXXXX ====1=====   3321 -119  (+1 -20 =49)
 8: Andscacs 0.94    3439  24.5 / 70   00=00000== 0=0=0===0= 0==0=0=0== 01====010= 0=0=01=00= =====1=0== ====0===== XXXXXXXXXX   3310 -126  (+4 -25 =41)
------------------------------------------------------------------------------------------------------------------------------------------
279 games: +79 -23 =177
I want to suggest the elo of LC0 somehow manages to match the elo of whatever it is playing against. Well, that's one "explanation" of its rather curious behaviour:

If one looks at the loss count of the second program, Houdini against the final ranking of its opponents, we get, as would be expected, an decreasing gradient: 5,1,0,0,0,0,0
Komodo gets: 2,3,2,0,1,0,0
Ethereal: 6,3,3,1,2,0,1
Fire: 6,6,3,3,1,0,1
Booot: 5,4,4,3,1,3,0
Andsacs: 7,4,4,3,5,1,1

Lc0 is different: 1,1,1,1,1,0,2, almost irrelevent who the opponent is, the loss rate remains almost constant.

Obviously, "non-losses", counting wins and draws together, shows the same pattern in reverse. Which suggests, well, to me, that LC0 doesn't really have an elo that can be mapped onto any particular opponent. It's not behaving itself properly according to the laws of elo ratings.

Robert Pope
Posts: 510
Joined: Sat Mar 25, 2006 7:27 pm

Re: Chess.com 2018 computer chess championship

Post by Robert Pope » Wed Sep 26, 2018 3:15 pm

chrisw wrote:
Wed Sep 26, 2018 9:32 am
I want to suggest the elo of LC0 somehow manages to match the elo of whatever it is playing against. Well, that's one "explanation" of its rather curious behaviour:

If one looks at the loss count of the second program, Houdini against the final ranking of its opponents, we get, as would be expected, an decreasing gradient: 5,1,0,0,0,0,0
Komodo gets: 2,3,2,0,1,0,0
Ethereal: 6,3,3,1,2,0,1
Fire: 6,6,3,3,1,0,1
Booot: 5,4,4,3,1,3,0
Andsacs: 7,4,4,3,5,1,1

Lc0 is different: 1,1,1,1,1,0,2, almost irrelevent who the opponent is, the loss rate remains almost constant.

Obviously, "non-losses", counting wins and draws together, shows the same pattern in reverse. Which suggests, well, to me, that LC0 doesn't really have an elo that can be mapped onto any particular opponent. It's not behaving itself properly according to the laws of elo ratings.
I think the issue is that LC0 plays very well, except for a glaring hole that almost any opponent can capitalize on. Other engines have their own holes, but they are subtle, so weaker engines are less likely to discover them.

If an engine were programmed to put its queen randomly en prise in 10% of games, it's going to lose at least 10% of the time, no matter who the opponent is.

Post Reply