LCZero Accomplishments and Goals Thus Far

[duncan]

1. To reach the level of A0 (estimated 3300 rating in GTX 1060)

2. To reach the level of latest stockfish on GTX 1060 ( estimated 3550+ rating on 8 cores desktop)

3. To surpass the rating of stockfish on GTX 1060.(getting 3600+)



Achievement milestones

1. Finished 10 millions games,

2. Reached 2800+, super GM level, on 1060 GTX.

3. Gofundme got €5000+ donation for the project.



Future Leela

1. Network will soon be expanded into 15x192 and may further expand into 20x256

( exactly as A0)

2. cuDNN or tensorflow implementation will increase the speed/ elo of Leela on NVIDIA cards (?50% ?100% ?200%), ( too bad for AMD cards though).

3. syzgy Tablebase

4. Auto resign will speed up training up to 30%.

It looks like ID 227 was trained ON NEW NETWORK 15x192! A HUGE jump in ELO just happened:http://lczero.org/ not sure though.

Yes!! Tablebases are coming!

I don't think tablebases should be used in the training of course.

https://www.chess.com/forum/view/genera ... ro?page=13

drmrboss

[duncan]

It still blunders hugely after the +300 elo.

It does, no question about it. Even with the new cuDNN speedups (which need testing to see how that translates into results). It is a fascinating engine, but there is a ways to go yet.

[duncan]

You guys are pretty funny, +300 elo since ID125 right?? Lets see how this short test games ends first...

so 300 elo speedup is not possible since ID125 . too good to be true?

[Guenther]

You probably meant 227. ID 232 is MUCH stronger.

I also think he meant ID 227. I don't understand though, why you believe ID232 should be much better than ID227?

It is not even clear to me that the latest networks are currently already better than previous 10*128 NNs.
I hope you noticed that the SP rating is now completely obsolete in comparison to previous NNs,
because nodes and not time controls are used for measurement, while the newer 15*192 NNs dropped nps by 50%.
(which means it needs now twice the time to get the same nodes, which is not reflected in the graph and makes most of the 100+ 'jump' in SP rating).

Here is my current testing with huge error bars for ID231, which had just started a few hours ago.

Code: Select all

   # PLAYER             :   RATING   ERROR  POINTS  PLAYED   (%)
   1 Glaurung_201-64    :  2745.23   78.06    44.0      62  71.0
   2 Rodent_10-64       :  2697.34   79.34    40.5      62  65.3
   3 LCZero_07ID231     :  2659.17  132.50    13.5      23  58.7
   4 LCZero_07ID181     :  2653.98   38.60   174.0     300  58.0
   5 Rotor_08           :  2611.81   76.35    34.5      64  53.9
   6 Danasah_70         :  2610.21   75.51    33.5      62  54.0
   7 Jellyfish_11-64    :  2599.77   78.86    33.0      63  52.4
   8 Tucano_400-64      :  2598.21   75.29    32.5      62  52.4
   9 Chronos_197        :  2568.31   71.96    30.0      62  48.4
  10 Monolith_04-64     :  2538.26   80.42    27.5      62  44.4
  11 Counter_12-64      :  2501.46   82.95    24.5      62  39.5
  12 LCZero_07ID150     :  2501.13   37.25   113.0     300  37.7
  13 Hermann_25-64      :  2476.14   79.60    22.5      62  36.3

White advantage = 34.41 +/- 13.04
Draw rate (equal opponents) = 21.73 % +/- 1.79

[CMCanavessi]

Well, 232 is doing much better at the CCLS gauntlet than what 227 did. Maybe just luck for now (it's about to get to half-point), we'll see how it ends but it's looking good.

[Nay Lin Tun]

1. To reach the level of A0 (estimated 3300 rating in GTX 1060)

2. To reach the level of latest stockfish on GTX 1060 ( estimated 3550+ rating on 8 cores desktop)

3. To surpass the rating of stockfish on GTX 1060.(getting 3600+)



Achievement milestones

1. Finished 10 millions games,

2. Reached 2800+, super GM level, on 1060 GTX.

3. Gofundme got €5000+ donation for the project.



Future Leela

1. Network will soon be expanded into 15x192 and may further expand into 20x256

( exactly as A0)

2. cuDNN or tensorflow implementation will increase the speed/ elo of Leela on NVIDIA cards (?50% ?100% ?200%), ( too bad for AMD cards though).

3. syzgy Tablebase

4. Auto resign will speed up training up to 30%.

It looks like ID 227 was trained ON NEW NETWORK 15x192! A HUGE jump in ELO just happened:http://lczero.org/ not sure though.

Yes!! Tablebases are coming!

I don't think tablebases should be used in the training of course.

https://www.chess.com/forum/view/genera ... ro?page=13

drmrboss

Yeah, it is my nick in chess.com. Yeah I posted this in here, lcgoogle group and chess.com.

[Guenther]

Here is my current testing with huge error bars for ID231, which had just started a few hours ago.

...

There was a slight error in calibrating due to a previous adjustment value
in my ordo batch file from another run (15 too low in comparison to opp. candidates)

Code: Select all

#       PLAYER          RATING  ERROR   POINTS  PLAYED  (%)     CCRL Rating     Diff
01      Glaurung_201-64 2760.46  84.57   44.0    62     71.0    2740             20.46
02      Rodent_10-64    2712.59  80.06   40.5    62     65.3    2692             20.59
03      LCZero_07ID181  2669.45  34.66  174.0   300     58.0    *               *
04      LCZero_07ID231  2668.42 114.84   14.5    25     58.0    *               *       
05      Rotor_08        2626.89  70.77   34.5    64     53.9    2612             14.89
06      Danasah_70      2625.48  75.92   33.5    62     54.0    2596             29.48
07      Jellyfish_11-64 2615.27  77.01   33.5    64     52.3    2608              7.27
08      Tucano_400-64   2613.49  78.84   32.5    62     52.4    2662            -48.51
09      Chronos_197     2585.52  76.54   30.5    63     48.4    2639            -53.48
10      Monolith_04-64  2553.56  80.06   27.5    62     44.4    2597            -43.44
11      Counter_12-64   2516.77  81.28   24.5    62     39.5    2446             70.77
12      LCZero_07ID150  2516.63  36.14  113.0   300     37.7    *               *
13      Hermann_25-64   2491.46  78.85   22.5    62     36.3    2512            -20.54
--------------------------------------------------------------------------------------
Gauntlet Opp Rating     2610.15                                 2610.4          -2.51

[CMCanavessi]

Here's my current list (232 gauntlet ongoing):

Code: Select all

   # PLAYER                                  :  RATING  PLAYED    W    D    L   (%)  D(%)  OppAvg  OppN  OppDiv
  93 Leela Chess Zero v0.7 ID 232 x64        :  2698.6     100   48   21   31    59    21  2635.6    25    25.0
  97 Leela Chess Zero v0.7 ID 195 x64        :  2666.9     200   86   45   69    54    23  2635.6    25    25.0
 101 Leela Chess Zero v0.7 ID 227 x64        :  2655.9     200   77   57   66    53    29  2635.6    25    25.0
 102 Leela Chess Zero v0.7 ID 219 x64        :  2654.0     200   74   62   64    53    31  2635.6    25    25.0
 108 Leela Chess Zero v0.7 ID 210 x64        :  2635.7     200   71   58   71    50    29  2635.6    25    25.0
 115 Leela Chess Zero v0.7 ID 189 x64        :  2604.4     200   61   61   78    46    31  2635.6    25    25.0
 119 Leela Chess Zero v0.7 ID 185 x64        :  2572.7     200   58   50   92    42    25  2635.6    25    25.0
 127 Leela Chess Zero v0.7 ID 176 x64        :  2521.6     400  188   80  132    57    20  2455.5    50    50.0
 132 Leela Chess Zero v0.7 ID 162 x64        :  2428.9     200  119   40   41    70    20  2275.4    25    25.0
 133 Leela Chess Zero v0.7 ID 169 x64        :  2420.2     200  118   38   44    69    19  2275.4    25    25.0
 136 Leela Chess Zero v0.7 ID 156 x64        :  2411.8     200  115   40   45    68    20  2275.4    25    25.0
 139 Leela Chess Zero v0.7 ID 150 x64        :  2367.1     200  104   40   56    62    20  2275.4    25    25.0
 147 Leela Chess Zero v0.5 ID 129 x64        :  2330.4     200  103   23   74    57    12  2275.4    25    25.0
 148 Leela Chess Zero v0.5 ID 124 x64        :  2328.5     200   95   38   67    57    19  2275.4    25    25.0
 149 Leela Chess Zero v0.6 ID 125 x64        :  2319.0     200   90   43   67    56    22  2275.4    25    25.0
 152 Leela Chess Zero v0.7 ID 145 x64        :  2300.2     200   87   39   74    53    20  2275.4    25    25.0
 162 Leela Chess Zero v0.6 ID 138 x64        :  2227.2     200   70   34   96    44    17  2275.4    25    25.0
 177 Leela Chess Zero v0.5 ID 123 x64        :  2177.7     400  215   58  127    61    15  2062.1    50    50.0
 202 Leela Chess Zero ID 122 x64             :  2050.6     200  136   24   40    74    12  1848.9    25    25.0
 227 Leela Chess Zero ID 103 x64             :  1950.3     200  113   25   62    63    13  1848.9    25    25.0
 232 Leela Chess Zero ID 116 x64             :  1921.8     200  107   23   70    59    12  1848.9    25    25.0
 235 Leela Chess Zero ID 98 x64              :  1900.0     200   99   28   73    57    14  1848.9    25    25.0
 251 Leela Chess Zero ID 80 x64              :  1806.3     200   76   26   98    45    13  1848.9    25    25.0
 265 Leela Chess Zero ID 69 x64              :  1739.7     200   62   21  117    36    11  1848.9    25    25.0
 283 Leela Chess Zero Gen 33 (ID 55) x64     :  1658.4     400  185   35  180    51     9  1632.6    45    43.5
 305 Leela Chess Zero Gen 30 (ID 44) x64     :  1495.8     200  106   22   72    59    11  1416.3    25    25.0
 313 Leela Chess Zero Gen 24 x64             :  1391.6     200   84   22   94    48    11  1416.3    25    25.0
 319 Leela Chess Zero Gen 20 x64             :  1308.5     200   64   27  109    39    14  1416.3    25    25.0
 331 Leela Chess Zero Gen 16 x64             :  1220.3     200   47   26  127    30    13  1416.3    25    25.0
 333 Leela Chess Zero Gen 14 x64             :  1176.5     200   42   20  138    26    10  1416.3    25    25.0
 337 Leela Chess Zero Gen 12 x64             :  1073.1     296   95   32  169    38    11  1174.8    49    46.3
 349 Leela Chess Zero Gen 10 x64             :   863.1      92   53   11   28    64    12   656.0    23    23.0
 352 Leela Chess Zero Gen 8 x64              :   794.5      92   45   17   30    58    18   656.0    23    23.0
 357 Leela Chess Zero Gen 6 x64              :   599.5      92   31   18   43    43    20   656.0    23    23.0
 361 Leela Chess Zero Gen 4 x64              :   368.8     150   43   18   89    35    12   621.3    15    15.0

[Milos]

It still blunders hugely after the +300 elo.

It does, no question about it. Even with the new cuDNN speedups (which need testing to see how that translates into results). It is a fascinating engine, but there is a ways to go yet.

With this sort of speedu-up due to cuDNN, at LTC, say 60'+ 15'', your LC0 should be about the level of Houdini 1.5a on 4 threads. It would be a fascinating match, H1.5 was a very tactically astute engine. In 10 games, there will be some LC0 wins, and I would be curious how LC0 missing mates in 3, and generally being very weak in tactics, especially in tactical puzzles, can beat H1.5, mostly positionally and on long term "plans". Seeing this cuDNN speed-up, I will get a GTX 1060 in less than 2 weeks.

You are greatly overestimating LC0 strength.
I run again 1000 games match between LC0 ID227 1000 playouts vs. SF depth 10, and LC0 didn't even reach 40%.
SF depth 10 is like 800-1000Elo weaker than H1.5 on 4 threads on 60'+15'' TC.
LC0 on cuDNN and 1060 is ~5000nps on 15x128 net, i.e. around 350'000 playouts per move on 60'+15'' TC.
If you really believe 8.5 doublings for LC0 is worth even close to 800 or 1000 Elo, what can I tell you, you are grossly mistaken. Just look at Andrey Chilantiev results in Tournaments section where it is clear than doubling in number of playouts for LC0 yields 50-60Elo.

[FWCC]

Top of the post courtesy to drmrboss