LCZero progress

majortom · Post by **majortom** » Fri May 18, 2018 11:21 pm

     Program                      Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 239_0df31cb9 N714    :  216.3   32.3   32.3   508    65.9 %     94   34.6 %
   2 LC0 302_ade7ac45 N714    :  188.8   44.9   44.9   164    55.8 %    142   45.7 %
   3 LC0 173_101eeb41 N2k     :  137.2   30.5   30.5   400    53.8 %    108   33.5 %
   4 LC0 125_738edc3d N2k     :    0.0    0.0    0.0   456    26.9 %    178   28.3 %

Code: Select all

1 LC0 239_0df31cb9 N714 :216.3 508 (+247,=176,- 85), 65.9 %

LC0 302_ade7ac45 N714   : 108 (+ 25,= 53,- 30), 47.7 %
LC0 125_738edc3d N2k    : 200 (+134,= 48,- 18), 79.0 %
LC0 173_101eeb41 N2k    : 200 (+ 88,= 75,- 37), 62.8 %

2 LC0 302_ade7ac45 N714 :188.8 164 (+ 54,= 75,- 35), 55.8 %

LC0 239_0df31cb9 N714   : 108 (+ 30,= 53,- 25), 52.3 %
LC0 125_738edc3d N2k    : 56 (+ 24,= 22,- 10), 62.5 %

3 LC0 173_101eeb41 N2k  :137.2 400 (+148,=134,-118), 53.8 %

LC0 239_0df31cb9 N714   : 200 (+ 37,= 75,- 88), 37.3 %
LC0 125_738edc3d N2k    : 200 (+111,= 59,- 30), 70.3 %

4 LC0 125_738edc3d N2k  :  0.0 456 (+ 58,=129,-269), 26.9 %

LC0 302_ade7ac45 N714   : 56 (+ 10,= 22,- 24), 37.5 %
LC0 239_0df31cb9 N714   : 200 (+ 18,= 48,-134), 21.0 %
LC0 173_101eeb41 N2k    : 200 (+ 30,= 59,-111), 29.8 %

Milos · Post by **Milos** » Sat May 19, 2018 1:37 am

majortom wrote: ↑Fri May 18, 2018 11:07 pm LCZero ID173 - 128 000 playouts per move vs. SF 130418 at 15 depth:

Code: Select all

   # PLAYER                    :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Stockfish 130418 D15      :   112.9   36.3   171.0     266    64
   2 LC0 173_101eeb41 N128k    :    37.0   68.4    26.0      66    39
   3 LC0 173_101eeb41 N64k     :     0.0   ----    69.0     200    35

White advantage = 25.93 +/- 16.91
Draw rate (equal opponents) = 50.00 % +/- 0.00

Thanks a lot for tests, these are by far the most elaborate scaling tests of LC0.
Even though the net is old (but not more than 50-60Elo weaker then the strongest network ever) it is really astonishing to see that LC0 even with 128k playouts per move can't win SFdev depth 15.
Running net 173 on a TPU wouldn't probably give more than those 128k nodes per second. OTOH SFdev needs like 50ms to reach depth 15 on a strong machine. So even with given TPU and 20x time odds LC0 wouldn't beat SFdev. So much about reaching A0 level...

epanek · Post by **epanek** » Sat May 19, 2018 2:17 am

How do you eat an elephant?

majortom · Post by **majortom** » Sat May 19, 2018 8:18 am

After 50% of these gauntlet games:

Code: Select all

   # PLAYER                   :  RATING  ERROR  POINTS  PLAYED   (%)
   1 LC0 239_0df31cb9 N714    :   212.9   31.0   345.0     529    65
   2 LC0 302_ade7ac45 N714    :   186.7   43.3   112.5     200    56
   3 LC0 173_101eeb41 N2k     :   135.4   30.5   215.0     400    54
   4 LC0 125_738edc3d N2k     :     0.0   ----   127.5     471    27

White advantage = 23.79 +/- 8.82
Draw rate (equal opponents) = 50.00 % +/- 0.00

Code: Select all

1 LC0 239_0df31cb9 N714 :212.9 529 (+252,=186,- 91), 65.2 %

LC0 302_ade7ac45 N714   : 129 (+ 30,= 63,- 36), 47.7 %
LC0 125_738edc3d N2k    : 200 (+134,= 48,- 18), 79.0 %
LC0 173_101eeb41 N2k    : 200 (+ 88,= 75,- 37), 62.8 %

2 LC0 302_ade7ac45 N714 :186.7 200 (+ 67,= 91,- 42), 56.3 %

LC0 239_0df31cb9 N714   : 129 (+ 36,= 63,- 30), 52.3 %
LC0 125_738edc3d N2k    : 71 (+ 31,= 28,- 12), 63.4 %

3 LC0 173_101eeb41 N2k  :135.4 400 (+148,=134,-118), 53.8 %

LC0 239_0df31cb9 N714   : 200 (+ 37,= 75,- 88), 37.3 %
LC0 125_738edc3d N2k    : 200 (+111,= 59,- 30), 70.3 %

4 LC0 125_738edc3d N2k  :  0.0 471 (+ 60,=135,-276), 27.1 %

LC0 302_ade7ac45 N714   : 71 (+ 12,= 28,- 31), 36.6 %
LC0 239_0df31cb9 N714   : 200 (+ 18,= 48,-134), 21.0 %
LC0 173_101eeb41 N2k    : 200 (+ 30,= 59,-111), 29.8 %

majortom · Post by **majortom** » Sat May 19, 2018 10:54 pm

Code: Select all

   # PLAYER                   :  RATING  ERROR  POINTS  PLAYED   (%)
   1 LC0 239_0df31cb9 N714    :   207.7   29.3   363.5     570    64
   2 LC0 302_ade7ac45 N714    :   189.5   39.3   148.5     261    57
   3 LC0 173_101eeb41 N2k     :   132.7   30.6   215.0     400    54
   4 LC0 125_738edc3d N2k     :     0.0   ----   134.0     491    27

White advantage = 20.43 +/- 8.70
Draw rate (equal opponents) = 50.00 % +/- 0.00

Code: Select all

Individual statistics:

1 LC0 239_0df31cb9 N714 :207.7 570 (+262,=203,-105), 63.8 %

LC0 302_ade7ac45 N714   : 170 (+ 40,= 80,- 50), 47.1 %
LC0 125_738edc3d N2k    : 200 (+134,= 48,- 18), 79.0 %
LC0 173_101eeb41 N2k    : 200 (+ 88,= 75,- 37), 62.8 %

2 LC0 302_ade7ac45 N714 :189.5 261 (+ 90,=117,- 54), 56.9 %

LC0 239_0df31cb9 N714   : 170 (+ 50,= 80,- 40), 52.9 %
LC0 125_738edc3d N2k    : 91 (+ 40,= 37,- 14), 64.3 %

3 LC0 173_101eeb41 N2k  :132.7 400 (+148,=134,-118), 53.8 %

LC0 239_0df31cb9 N714   : 200 (+ 37,= 75,- 88), 37.3 %
LC0 125_738edc3d N2k    : 200 (+111,= 59,- 30), 70.3 %

4 LC0 125_738edc3d N2k  :  0.0 491 (+ 62,=144,-285), 27.3 %

LC0 302_ade7ac45 N714   : 91 (+ 14,= 37,- 40), 35.7 %
LC0 239_0df31cb9 N714   : 200 (+ 18,= 48,-134), 21.0 %
LC0 173_101eeb41 N2k    : 200 (+ 30,= 59,-111), 29.8 %

majortom · Post by **majortom** » Sat May 19, 2018 10:58 pm

Code: Select all

   # PLAYER                    :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Stockfish 130418 D15      :   112.9   35.7   182.0     289    63
   2 LC0 173_101eeb41 N128k    :    59.6   58.7    38.0      89    43
   3 LC0 173_101eeb41 N64k     :     0.0   ----    69.0     200    35

White advantage = 25.91 +/- 15.25
Draw rate (equal opponents) = 50.00 % +/- 0.00

Code: Select all

1 Stockfish 130418 D15   :112.9 289 (+136,= 92,- 61), 63.0 %

LC0 173_101eeb41 N64k    : 200 (+ 98,= 66,- 36), 65.5 %
LC0 173_101eeb41 N128k   : 89 (+ 38,= 26,- 25), 57.3 %

2 LC0 173_101eeb41 N128k : 59.6 89 (+ 25,= 26,- 38), 42.7 %

Stockfish 130418 D15     : 89 (+ 25,= 26,- 38), 42.7 %

3 LC0 173_101eeb41 N64k  :  0.0 200 (+ 36,= 66,- 98), 34.5 %

Stockfish 130418 D15     : 200 (+ 36,= 66,- 98), 34.5 %

majortom · Post by **majortom** » Sun May 20, 2018 6:31 pm

Code: Select all

   # PLAYER                    :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Stockfish 130418 D15      :   113.1   36.9   190.0     303    63
   2 LC0 173_101eeb41 N128k    :    59.3   56.0    44.0     103    43
   3 LC0 173_101eeb41 N64k     :     0.0   ----    69.0     200    35

White advantage = 30.05 +/- 15.06
Draw rate (equal opponents) = 50.00 % +/- 0.00

majortom · Post by **majortom** » Sun May 20, 2018 9:25 pm

SF D15 was mated in 33 moves:
https://lichess.org/muczG7mb/black

Here you can find other LCZero vs. SF games:
https://lichess.org/@/maj-or-tom/import

majortom · Post by **majortom** » Tue May 22, 2018 12:29 am

Code: Select all

   # PLAYER                    :  RATING  ERROR  POINTS  PLAYED   (%)
   1 Stockfish 130418 D15      :   112.8   37.2   200.5     323    62
   2 LC0 173_101eeb41 N128k    :    65.9   55.9    53.5     123    43
   3 LC0 173_101eeb41 N64k     :     0.0   ----    69.0     200    35

White advantage = 21.72 +/- 14.61
Draw rate (equal opponents) = 50.00 % +/- 0.00

Code: Select all

1 Stockfish 130418 D15   :112.8 323 (+150,=101,- 72), 62.1 %

LC0 173_101eeb41 N64k    : 200 (+ 98,= 66,- 36), 65.5 %
LC0 173_101eeb41 N128k   : 123 (+ 52,= 35,- 36), 56.5 %

2 LC0 173_101eeb41 N128k : 65.9 123 (+ 36,= 35,- 52), 43.5 %

Stockfish 130418 D15     : 123 (+ 36,= 35,- 52), 43.5 %

3 LC0 173_101eeb41 N64k  :  0.0 200 (+ 36,= 66,- 98), 34.5 %

Stockfish 130418 D15     : 200 (+ 36,= 66,- 98), 34.5 %

majortom · Post by **majortom** » Tue May 22, 2018 12:38 am

Code: Select all

   # PLAYER                   :  RATING  ERROR  POINTS  PLAYED   (%)
   1 LC0 302_ade7ac45 N714    :    15.8   33.3   104.5     200    52
   2 LC0 239_0df31cb9 N714    :     0.0   ----    95.5     200    48

White advantage = 19.34 +/- 18.51
Draw rate (equal opponents) = 50.00 % +/- 0.00

Games

LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress

Re: LCZero progress