TalkChess.com

George Tsavdaris wrote: ↑Fri Jul 13, 2018 11:10 pm

Laskos wrote: ↑Fri Jul 13, 2018 10:53 pm All in all, about 3310 CCRL performance at 5' + 5''. Your GPU is maybe 30-40% faster than my GTX 1060,

Actually it's about 100% faster (i.e x2) for running Leela as per a chart one of the devs of Leela has made:

i'm skeptical about how that translates; i run a 1080, typically @ 2050-2075 core & a very unscientific estimate of my nps from start position, in a fast TC game, is 11kn/s (mainserver net). i believe it goes up a few kn/s with time before hitting a wall. i have no idea, but i think 1060s are easily above ~6 kn/s (or whatever...not a math guy) at the same depth. i don't recall anymore what gave me that impression.

yanquis1972 wrote: ↑Sat Jul 14, 2018 2:34 am

George Tsavdaris wrote: ↑Fri Jul 13, 2018 11:10 pm

Laskos wrote: ↑Fri Jul 13, 2018 10:53 pm All in all, about 3310 CCRL performance at 5' + 5''. Your GPU is maybe 30-40% faster than my GTX 1060,

Actually it's about 100% faster (i.e x2) for running Leela as per a chart one of the devs of Leela has made:

i'm skeptical about how that translates; i run a 1080, typically @ 2050-2075 core & a very unscientific estimate of my nps from start position, in a fast TC game, is 11kn/s (mainserver net). i believe it goes up a few kn/s with time before hitting a wall. i have no idea, but i think 1060s are easily above ~6 kn/s (or whatever...not a math guy) at the same depth. i don't recall anymore what gave me that impression.

We can do controlled measures.

From starting position with 3 different net sizes and IDs here is what i get with 1070 Ti:

Lc0 ID125 10x128 Lc0 ID476 15x192 Lc0 Test10-10047 20x256 Perhaps you with your 1080 and Kai with 1060 6 GB can run the 15x192 ID476 net and compare. Or the other 2 nets also....

George Tsavdaris wrote: ↑Sat Jul 14, 2018 8:17 am

yanquis1972 wrote: ↑Sat Jul 14, 2018 2:34 am

George Tsavdaris wrote: ↑Fri Jul 13, 2018 11:10 pm

Laskos wrote: ↑Fri Jul 13, 2018 10:53 pm All in all, about 3310 CCRL performance at 5' + 5''. Your GPU is maybe 30-40% faster than my GTX 1060,

Actually it's about 100% faster (i.e x2) for running Leela as per a chart one of the devs of Leela has made:

i'm skeptical about how that translates; i run a 1080, typically @ 2050-2075 core & a very unscientific estimate of my nps from start position, in a fast TC game, is 11kn/s (mainserver net). i believe it goes up a few kn/s with time before hitting a wall. i have no idea, but i think 1060s are easily above ~6 kn/s (or whatever...not a math guy) at the same depth. i don't recall anymore what gave me that impression.

We can do controlled measures.

From starting position with 3 different net sizes and IDs here is what i get with 1070 Ti:

Lc0 ID476 15x192

Code: Select all

2/27    00:14     168,408    11,594    +0,16    c2-c4 Ng8-f6 Nb1-c3 e7-e5 Ng1-f3 Nb8-c6 g2-g3 d7-d5 c4xd5 Nf6xd5 Bf1-g2 Nd5-b6 0-0 Bf8-e7 

Same ID, GTX 1060 6GB, to the same number of nodes: So, 1070 Ti is 62% faster than 1060, not exactly 100%. GTX 1080 shouldn't be much faster than 1070 Ti.

Laskos wrote: ↑Fri Jul 13, 2018 9:26 pm

Michel wrote: ↑Fri Jul 13, 2018 9:01 pm Not sure if the result against K12.1 is really disappointing. The error bars indicate that both engines could still be equal strength...

Not sure about those error bars, in any case, for skewed results, they should be asymmetrical. LOS with a uniform prior is easy to get, and it is 96.5%. Not very encouraging. LOS with a bit informed prior can be tamed somewhat, with a rough prior (4 * score * (1-score))^4, which basically assumes that the difference is hardly above 200 Elo points, I can get a LOS of 93.6%, still not very nice. I will maybe run another 10 games overnight, if something similar happens, it will be much clearer.

The second overnight match went +3 -0 =7 for Komodo 12.1.1, a bit different, but still bad. Totally in 20 games at 600'' + 10'':

TC: 600'' + 10''
Komodo 12.1.1 vs Lc0_11jul ID482
+9 -1 =10 [0.700] 20
Elo difference: 147.19
errors 95% confidence:
-96.65
+125.54

Performance: 3270 CCRL 40/4' Elo

LOS 99.4%
LOS p1 99.0%
LOS p2 98.6%

I calculated myself the 95% confidence error bars from the scratch, but assuming normal distribution (which is a bit awkward here). It is pretty clear then that Leela is at least 50 Elo points weaker in these condition than Komodo, and has a meager expected performance of 3270 CCRL Elo points. It seems to not scale significantly better than Komodo from 60'' + 1'' to 600'' + 10''. A bit disappointing. Uninformed LOS of Komodo is 99.4%, pretty decisive. With a prior p1, basically saying that engines are hardly separated by more than 200 Elo points, LOS doesn't decrease significantly, to 99.0%. WIth even a stronger prior, that engines are hardly separated by more than 100 Elo points, LOS is 98.6%. Such stability is due to the relative large number of draws.

There are reasons of worry for WCCC. George's results show a similar meager performance at 300'' + 5'' in many games. Although TCEC results on 2x GTX 1080 Ti are encouraging, I saw that Leela NPS in TCEC now are only some 2 times lower than what 8 x V100 reported, which is a surprise to me, I expected a factor of 4 maybe. If Leela doesn't scale well to LTC (as I assumed it would), then there is a little chance it to perform at Komodo 12+ level on 48 cores. Let's see.

Laskos wrote: ↑Sat Jul 14, 2018 11:11 am

Laskos wrote: ↑Fri Jul 13, 2018 9:26 pm

Michel wrote: ↑Fri Jul 13, 2018 9:01 pm Not sure if the result against K12.1 is really disappointing. The error bars indicate that both engines could still be equal strength...

Not sure about those error bars, in any case, for skewed results, they should be asymmetrical. LOS with a uniform prior is easy to get, and it is 96.5%. Not very encouraging. LOS with a bit informed prior can be tamed somewhat, with a rough prior (4 * score * (1-score))^4, which basically assumes that the difference is hardly above 200 Elo points, I can get a LOS of 93.6%, still not very nice. I will maybe run another 10 games overnight, if something similar happens, it will be much clearer.

The second overnight match went +3 -0 =7 for Komodo 12.1.1, a bit different, but still bad. Totally in 20 games at 600'' + 10'':

TC: 600'' + 10''
Komodo 12.1.1 vs Lc0_11jul ID482
+9 -1 =10 [0.700] 20
Elo difference: 147.19
errors 95% confidence:
-96.65
+125.54

Performance: 3270 CCRL 40/4' Elo

LOS 99.4%
LOS p1 99.0%
LOS p2 98.6%

I calculated myself the 95% confidence error bars from the scratch, but assuming normal distribution (which is a bit awkward here). It is pretty clear then that Leela is at least 50 Elo points weaker in these condition than Komodo, and has a meager expected performance of 3270 CCRL Elo points. It seems to not scale significantly better than Komodo from 60'' + 1'' to 600'' + 10''. A bit disappointing. Uninformed LOS of Komodo is 99.4%, pretty decisive. With a prior p1, basically saying that engines are hardly separated by more than 200 Elo points, LOS doesn't decrease significantly, to 99.0%. WIth even a stronger prior, that engines are hardly separated by more than 100 Elo points, LOS is 98.6%. Such stability is due to the relative large number of draws.

There are reasons of worry for WCCC. George's results show a similar meager performance at 300'' + 5'' in many games. Although TCEC results on 2x GTX 1080 Ti are encouraging, I saw that Leela NPS in TCEC now are only some 2 times lower than what 8 x V100 reported, which is a surprise to me, I expected a factor of 4 maybe. If Leela doesn't scale well to LTC (as I assumed it would), then there is a little chance it to perform at Komodo 12+ level on 48 cores. Let's see.

Leela doesn‘t know how to handle increment time.
Often she ends up with 3x-5x more base time, at the end of the games, due to bad increment management.
And this looks to be true. To be safe I‘ve checked it myself with 800 games / 15 seconds per move.
It‘s like: nice I will get increment time with every move...what should I do with it...I don’t know...I will add it to my base time.
This will be a super big problem at TCEC, different divisions with different increments, maybe something like: +10, +15, +30, +45, +60, +90, +120, +180 seconds per move!
Most increment improvements are done to improve play with only +1 to +5 seconds per move and that‘s a big mistake.
The bigger an increment is, the bigger is the base time at the and. I had seen lots of game which started with +5 minutes base time and ended with +95 minutes base time .

re lc0 & scaling, check out this chart: https://docs.google.com/spreadsheets/d/ ... =583567367

2 things to be aware of: 1) "nodes/move was from a middlegame position", which isn't given afaik. 2) june 4th lc0.exe used

(and, obviously, not finished yet)

potentially little to no gain after as little as 30s/move average hardware? which is somehow not dissimilar to A0 (by my interpretation) with a different net size on completely different hardware.

A taste of what could happen in the next days at WCCC. 4 LTC games, side and reversed from 2 starting positions. According to my estimates, at this long time control, 60 minutes + 60 seconds increment, lc0 ID491 on GTX 1060 GPU should be a bit stronger than the latest Komodo 12.1.1 on one i7 core from regular, quiet starting positions.

Games: 3600'' + 60''

From standard opening position, 2 draws:

[pgn][Event "My Tournament"] [Site "?"] [Date "2018.07.14"] [Round "1"] [White "Komodo 12.1.1"] [Black "lc0 ID491"] [Result "1/2-1/2"] [FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"] [GameDuration "05:36:53"] [GameEndTime "2018-07-14T18:46:19.319 GTB Daylight Time"] [GameStartTime "2018-07-14T13:09:25.964 GTB Daylight Time"] [PlyCount "221"] [SetUp "1"] [TimeControl "3600+60"] 1. d4 {+0.33/28 209s} Nf6 {-0.10/3 202s} 2. c4 {+0.36/29 178s} e6 {-0.10/3 121s} 3. Nf3 {+0.38/29 147s} d5 {-0.07/4 163s} 4. Nc3 {+0.37/29 175s} dxc4 {-0.04/3 175s} 5. e4 {+0.48/29 124s} Bb4 {-0.04/4 96s} 6. Bg5 {+0.44/30 167s} c5 {-0.01/3 111s} 7. Bxc4 {+0.40/29 122s} h6 {0.00/4 42s} 8. Bxf6 {+0.29/28 141s} Qxf6 {0.00/4 25s} 9. e5 {+0.29/30 151s} Qd8 {0.00/4 31s} 10. O-O {+0.34/30 159s} cxd4 {+0.22/3 201s} 11. Nxd4 {+0.22/30 139s} O-O {+0.17/3 132s} 12. Rc1 {+0.36/30 81s} Bd7 {+0.31/3 175s} 13. Qg4 {+0.31/29 160s} Qc7 {+0.26/3 168s} 14. Nf3 {+0.34/29 134s} Bc6 {+0.14/3 148s} 15. Ne4 {+0.29/31 117s} Nd7 {+0.16/3 104s} 16. Nf6+ {+0.27/31 143s} Kh8 {+0.16/3 35s} 17. Nxd7 {+0.23/32 132s} Qxd7 {+0.18/3 66s} 18. Rfd1 {+0.20/31 144s} Qe7 {+0.18/3 41s} 19. Nd4 {+0.19/31 127s} Be8 {+0.19/3 91s} 20. h4 {+0.09/29 192s} Bc5 {+0.39/3 221s} 21. Bb3 {+0.11/32 189s} Rd8 {+0.44/3 356s} 22. Rc3 {0.00/32 140s} a5 {+0.52/3 206s} 23. Rcd3 {+0.11/32 100s} Bb6 {+0.56/3 196s} 24. Qe4 {+0.04/32 128s} Bxd4 {+0.28/3 310s} 25. Rxd4 {+0.11/32 53s} Bc6 {+0.29/3 105s} 26. Qf4 {+0.12/34 121s} Rxd4 {+0.28/3 55s} 27. Rxd4 {0.00/33 120s} Rc8 {+0.26/3 56s} 28. Kh2 {0.00/31 104s} Qc7 {+0.15/3 279s} 29. Rd6 {+0.03/33 463s} Kg8 {+0.25/3 221s} 30. a3 {0.00/32 163s} Qb6 {+0.20/3 280s} 31. Bc4 {+0.09/32 81s} Re8 {+0.17/3 230s} 32. Qd4 {+0.07/34 124s} Qxd4 {0.00/3 158s} 33. Rxd4 {0.00/33 52s} Kf8 {-0.01/3 142s} 34. g4 {+0.08/32 69s} Rc8 {0.00/3 176s} 35. Kg3 {+0.01/33 110s} Ke7 {0.00/3 82s} 36. h5 {+0.06/33 108s} f6 {-0.02/3 238s} 37. f4 {0.00/34 60s} Bd7 {-0.07/3 135s} 38. exf6+ {+0.05/31 98s} gxf6 {-0.08/4 123s} 39. Re4 {0.00/33 99s} Rc5 {-0.09/3 97s} 40. b4 {+0.04/34 72s} axb4 {-0.13/4 40s} 41. axb4 {+0.04/34 30s} Rc6 {-0.12/3 57s} 42. Kh4 {+0.03/36 63s} Rd6 {-0.27/3 144s} 43. b5 {0.00/38 66s} b6 {-0.32/3 19s} 44. Kg3 {+0.02/37 75s} Be8 {-0.39/3 80s} 45. Re2 {+0.19/33 60s} Kf8 {-0.46/3 65s} 46. Re3 {+0.19/33 74s} Ke7 {-0.74/3 62s} 47. f5 {+0.18/35 64s} e5 {-0.47/3 50s} 48. Ra3 {+0.15/35 59s} Rd4 {-0.51/3 43s} 49. Be2 {+0.12/33 123s} Bf7 {-0.47/3 59s} 50. Ra7+ {+0.11/34 88s} Rd7 {-0.37/3 59s} 51. Ra6 {+0.09/36 58s} Rd6 {-0.33/3 51s} 52. Kf3 {+0.08/37 63s} Bb3 {-0.50/3 89s} 53. Ke4 {+0.07/37 61s} Bf7 {-0.51/3 74s} 54. Ke3 {+0.07/40 71s} Bd5 {-0.49/3 30s} 55. Bd3 {+0.07/36 65s} Bf7 {-0.47/3 67s} 56. Ra7+ {+0.04/36 83s} Rd7 {-0.39/3 57s} 57. Ra6 {+0.04/37 52s} Rd6 {-0.45/3 59s} 58. Ra8 {+0.04/38 61s} Rd8 {-0.31/3 61s} 59. Ra4 {+0.04/39 58s} Rd4 {-0.25/3 65s} 60. Rxd4 {0.00/37 65s} exd4+ {+0.02/4 57s} 61. Kxd4 {0.00/54 27s} Bb3 {+0.02/4 42s} 62. Ke3 {0.00/58 76s} Kd6 {+0.27/3 69s} 63. Kf4 {0.00/58 59s} Bd1 {+0.27/3 51s} 64. Bf1 {0.00/53 59s} Ke7 {+0.30/4 77s} 65. Bg2 {0.00/57 68s} Be2 {+0.40/4 60s} 66. Bc6 {0.00/54 76s} Kf8 {+0.40/4 66s} 67. Kg3 {0.00/58 61s} Kf7 {+0.39/4 58s} 68. Kf4 {0.00/66 53s} Kg8 {+0.39/4 106s} 69. Be8 {0.00/56 54s} Kg7 {+0.39/3 41s} 70. Bc6 {0.00/59 56s} Kf8 {+0.39/3 81s} 71. Kg3 {0.00/64 72s} Ke7 {+0.39/4 20s} 72. Kh4 {0.00/63 63s} Kd6 {+0.39/4 75s} 73. Kh3 {0.00/65 259s} Ke5 {+0.40/4 91s} 74. Kg3 {0.00/65 53s} Bd1 {+0.39/4 35s} 75. Bf3 {0.00/51 46s} Bb3 {+0.37/4 42s} 76. Bc6 {0.00/72 49s} Ba4 {+0.38/4 81s} 77. Be8 {0.00/57 57s} Bc2 {+0.37/4 57s} 78. Bf7 {0.00/62 49s} Bd3 {+0.36/4 73s} 79. Be8 {0.00/77 51s} Bc2 {+0.36/3 44s} 80. Bd7 {0.00/64 58s} Bd1 {+0.36/4 83s} 81. Be8 {0.00/62 51s} Ba4 {+0.36/3 60s} 82. Bd7 {0.00/53 55s} Bc2 {+0.35/3 56s} 83. Bc6 {0.00/63 67s} Bb3 {+0.34/3 60s} 84. Bd7 {0.00/62 58s} Bd1 {+0.34/4 59s} 85. Be8 {0.00/72 76s} Kd6 {+0.34/3 57s} 86. Kf4 {0.00/60 61s} Ke7 {+0.31/4 64s} 87. Bc6 {0.00/63 117s} Be2 {+0.31/4 15s} 88. Kg3 {0.00/65 50s} Kf8 {+0.30/4 46s} 89. Kf4 {0.00/66 52s} Kf7 {+0.28/4 94s} 90. Bd7 {0.00/63 46s} Ke7 {+0.26/4 79s} 91. Bc6 {0.00/55 55s} Kd8 {+0.26/4 41s} 92. Kg3 {0.00/65 113s} Kc7 {+0.25/4 45s} 93. Kf4 {0.00/42 58s} Kd6 {+0.24/3 54s} 94. Kg3 {0.00/41 45s} Ke5 {+0.25/4 101s} 95. Be8 {0.00/70 47s} Bd1 {+0.25/3 67s} 96. Bg6 {0.00/72 50s} Kd5 {+0.27/3 49s} 97. Kf4 {0.00/49 46s} Kd6 {+0.23/4 55s} 98. Bf7 {0.00/57 62s} Kd7 {+0.24/4 79s} 99. Bc4 {0.00/56 53s} Ke8 {+0.24/3 51s} 100. Kg3 {0.00/63 47s} Ke7 {+0.25/4 46s} 101. Bf1 {0.00/61 47s} Kd7 {+0.23/4 71s} 102. Bg2 {0.00/62 48s} Be2 {+0.22/4 51s} 103. Bc6+ {0.00/63 58s} Kd6 {+0.20/4 53s} 104. Be8 {0.00/68 68s} Bd1 {+0.20/4 90s} 105. Kf4 {0.00/72 53s} Ba4 {+0.18/4 59s} 106. Bc6 {0.00/58 51s} Ke7 {+0.09/3 42s} 107. Ke3 {0.00/62 83s} Kf7 {+0.02/4 76s} 108. Kd2 {0.00/70 48s} Ke7 {+0.01/4 64s} 109. Ke2 {0.00/75 48s} Kd6 {0.00/4 54s} 110. Ke3 {0.00/73 238s} Bd1 {0.00/4 61s} 111. Kf4 {0.00/82 42s, Draw by fifty moves rule} 1/2-1/2[/pgn]

[pgn][Event "My Tournament"] [Site "?"] [Date "2018.07.14"] [Round "2"] [White "lc0 ID491"] [Black "Komodo 12.1.1"] [Result "1/2-1/2"] [FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"] [GameDuration "06:32:38"] [GameEndTime "2018-07-15T01:18:58.065 GTB Daylight Time"] [GameStartTime "2018-07-14T18:46:19.352 GTB Daylight Time"] [PlyCount "282"] [SetUp "1"] [Termination "adjudication"] [TimeControl "3600+60"] 1. c4 {+0.13/4 320s} e5 {-0.01/28 130s} 2. g3 {+0.16/3 209s} c6 {0.00/29 291s} 3. Nf3 {+0.23/3 182s} e4 {-0.08/30 141s} 4. Nd4 {+0.19/4 170s} d5 {+0.03/30 133s} 5. cxd5 {+0.18/3 79s} Qxd5 {0.00/30 169s} 6. Nc2 {+0.19/4 106s} Qh5 {0.00/30 176s} 7. Nc3 {+0.19/3 138s} Nf6 {0.00/32 116s} 8. Ne3 {+0.22/3 79s} Bc5 {0.00/31 142s} 9. Qa4 {+0.31/3 171s} O-O {0.00/30 106s} 10. Nxe4 {+0.50/3 179s} Nxe4 {-0.02/32 135s} 11. Qxe4 {+0.52/3 85s} Be6 {-0.04/32 244s} 12. Bg2 {+0.50/3 110s} Na6 {0.00/31 111s} 13. Bf3 {+0.68/3 182s} Qg6 {-0.09/32 244s} 14. Qxg6 {+0.93/3 181s} fxg6 {-0.42/30 119s} 15. O-O {+1.18/3 179s} Rad8 {-0.54/32 181s} 16. Kg2 {+1.23/3 157s} Nb4 {-0.58/32 121s} 17. a3 {+1.35/3 162s} Bxe3 {-0.67/35 161s} 18. axb4 {+1.35/3 106s} Bb6 {-0.54/35 155s} 19. d3 {+1.52/3 151s} Rf5 {-0.65/33 190s} 20. Bd2 {+1.53/3 252s} a6 {-0.55/35 119s} 21. Be4 {+1.43/3 179s} Rb5 {-0.62/35 363s} 22. e3 {+1.45/3 246s} Bf5 {-0.60/33 189s} 23. Bc3 {+1.64/3 189s} Bc7 {-0.59/34 231s} 24. h4 {+1.64/3 132s} Kf7 {-0.53/35 76s} 25. f3 {+1.92/3 157s} h5 {-0.53/35 77s} 26. Rfd1 {+1.95/3 305s} Re8 {-0.53/37 83s} 27. Kf2 {+1.92/3 133s} Bd8 {-0.53/39 90s} 28. Ra4 {+1.95/3 187s} Bc7 {-0.53/38 85s} 29. Re1 {+1.94/3 135s} Re7 {-0.53/36 77s} 30. Rg1 {+1.94/3 176s} Kg8 {-0.53/38 86s} 31. Rb1 {+1.96/3 270s} Kf7 {-0.54/41 104s} 32. Rd1 {+1.92/3 131s} Bb6 {-0.55/39 92s} 33. Rc1 {+1.94/3 80s} Bc7 {-0.56/40 102s} 34. Rg1 {+1.92/3 57s} Bb6 {-0.56/38 91s} 35. Ke2 {+1.90/3 54s} Bd8 {-0.56/37 94s} 36. Raa1 {+1.98/3 53s} Kg8 {-0.56/37 96s} 37. Raf1 {+2.04/3 75s} Rf7 {-0.45/33 104s} 38. Ra1 {+2.08/3 34s} Re7 {-0.56/37 68s} 39. Ra4 {+2.00/3 56s} Bc7 {-0.56/37 81s} 40. Kf2 {+1.91/3 47s} Kf7 {-0.56/36 106s} 41. Ke2 {+1.85/3 96s} Kg8 {-0.56/39 85s} 42. Ra3 {+1.93/3 42s} Kh7 {-0.56/38 111s} 43. Rg2 {+1.90/3 78s} Bh3 {-0.56/38 72s} 44. Rg1 {+1.91/3 33s} Bf5 {-0.56/39 73s} 45. Kf2 {+1.87/3 83s} Kg8 {-0.57/39 93s} 46. Rc1 {+1.84/3 52s} Kf7 {-0.55/37 71s} 47. Rg1 {+1.82/3 78s} Re8 {-0.56/37 61s} 48. Ra4 {+1.81/3 24s} Re7 {-0.56/37 65s} 49. Rga1 {+1.81/3 88s} Re8 {-0.56/37 68s} 50. Rf1 {+1.81/3 41s} Kg8 {-0.57/38 72s} 51. Rg1 {+1.80/3 68s} Re7 {-0.56/38 105s} 52. Ke2 {+1.85/3 81s} Re8 {-0.57/32 62s} 53. Raa1 {+1.81/3 42s} Re7 {-0.57/34 55s} 54. Kf2 {+1.78/3 56s} Re8 {-0.57/37 68s} 55. Ke2 {+1.76/3 79s} Re7 {-0.57/36 63s} 56. Bxf5 {+1.80/3 20s} gxf5 {-0.42/30 54s} 57. Rg2 {+1.79/3 36s} Kf8 {-0.41/29 59s} 58. d4 {+1.51/3 122s} Re6 {-0.43/31 91s} 59. Kd3 {+1.47/3 25s} Bd6 {-0.46/32 77s} 60. g4 {+1.52/3 55s} Rg6 {-0.46/33 91s} 61. g5 {+1.42/3 27s} Bxb4 {-0.35/28 35s} 62. e4 {+1.48/3 33s} Rd6 {-0.33/30 86s} 63. Ra4 {+1.56/3 53s} Bxc3 {-0.31/31 52s} 64. bxc3 {+1.61/3 52s} Rd8 {-0.44/32 92s} 65. Rb4 {+1.67/3 91s} Rxb4 {-0.28/33 78s} 66. cxb4 {+1.54/3 66s} fxe4+ {-0.25/33 82s} 67. fxe4 {+1.45/3 43s} Ke7 {-0.22/34 52s} 68. g6 {+1.40/3 32s} Rf8 {-0.24/33 56s} 69. Ke3 {+1.37/3 21s} Rf1 {-0.22/33 56s} 70. Rg5 {+1.69/3 102s} Re1+ {-0.29/30 56s} 71. Kf4 {+1.88/4 99s} Rf1+ {-0.02/32 42s} 72. Ke5 {+1.86/4 70s} Rf6 {-0.01/35 51s} 73. Rg1 {+1.80/4 122s} Rf2 {0.00/42 54s} 74. Rg3 {+1.79/3 57s} Rf1 {0.00/47 53s} 75. Rg2 {+1.77/3 48s} Rf8 {0.00/43 49s} 76. Rg1 {+1.79/4 87s} Rf3 {0.00/41 51s} 77. Rg2 {+1.78/3 50s} Rf8 {0.00/45 45s} 78. Rg3 {+1.75/3 59s} Rf1 {0.00/50 66s} 79. Rg5 {+1.74/3 36s} Rf6 {-0.02/35 101s} 80. Rxh5 {+1.37/3 32s} Rxg6 {-0.13/35 96s} 81. Rh7 {+1.16/3 35s} Re6+ {-0.11/33 61s} 82. Kf5 {+0.93/4 120s} Rf6+ {0.00/34 51s} 83. Ke5 {+0.93/4 0.41s} Re6+ {-0.07/34 55s} 84. Kf4 {+0.92/4 0.57s} Kf7 {-0.10/34 47s} 85. Rh8 {+0.85/3 87s} Re8 {-0.03/33 60s} 86. Rh5 {+1.77/3 80s} Re6 {0.00/32 52s} 87. Rh8 {+1.17/3 92s} Re8 {0.00/38 55s} 88. Rh5 {+1.13/3 37s} Re6 {0.00/34 55s} 89. e5 {+0.98/1001 44s} Rg6 {0.00/35 52s} 90. Rf5+ {+0.94/3 31s} Ke7 {0.00/37 52s} 91. h5 {+0.92/3 32s} Rg1 {0.00/36 63s} 92. Rg5 {+0.92/3 0.42s} Rf1+ {0.00/37 54s} 93. Kg4 {+0.97/4 212s} Ke6 {0.00/36 54s} 94. Rg6+ {+1.38/3 66s} Kd5 {0.00/38 51s} 95. Rd6+ {+1.38/3 68s} Ke4 {0.00/37 57s} 96. Kg3 {+1.24/3 32s} Rf3+ {0.00/38 51s} 97. Kg4 {+1.42/4 103s} Rf1 {0.00/38 55s} 98. Kg3 {+1.39/3 14s} Rf3+ {0.00/37 50s} 99. Kh4 {+0.83/4 78s} Rf1 {0.00/40 73s} 100. Rd8 {+0.71/4 126s} Rh1+ {0.00/36 60s} 101. Kg4 {+0.67/4 62s} Rg1+ {0.00/36 45s} 102. Kh3 {+0.64/4 28s} Kf5 {0.00/36 50s} 103. Rf8+ {+0.68/3 92s} Kg5 {0.00/38 60s} 104. Rf7 {+0.58/3 33s} Kxh5 {0.00/37 52s} 105. Rxb7 {+0.67/3 46s} Rd1 {0.00/37 46s} 106. Rd7 {+0.48/3 37s} g5 {0.00/37 54s} 107. Kg3 {+0.40/3 64s} g4 {0.00/39 57s} 108. Kf4 {+0.33/3 82s} Rf1+ {0.00/40 55s} 109. Ke4 {+0.18/4 87s} Re1+ {0.00/39 51s} 110. Kf5 {+0.13/4 66s} Rf1+ {0.00/41 77s} 111. Ke4 {+0.13/4 1.3s} Re1+ {0.00/45 49s} 112. Kf4 {+0.11/4 33s} Rf1+ {0.00/38 48s} 113. Ke3 {+0.11/4 53s} Re1+ {0.00/39 47s} 114. Kd2 {+0.10/4 116s} Re4 {0.00/39 57s} 115. Kd3 {+0.10/4 3.9s} Re1 {0.00/40 53s} 116. Rg7 {+0.08/4 115s} Kh4 {0.00/41 53s} 117. Kc4 {+0.07/4 53s} Rc1+ {0.00/39 53s} 118. Kb3 {+0.05/4 90s} Re1 {0.00/40 50s} 119. Kc4 {+0.04/3 68s} Rc1+ {0.00/44 53s} 120. Kd3 {+0.04/4 4.6s} Re1 {0.00/40 51s} 121. Rg6 {+0.03/3 67s} g3 {0.00/41 53s} 122. Kd2 {+0.02/4 117s} Re4 {0.00/41 71s} 123. Kd3 {+0.02/3 23s} Re1 {0.00/39 73s} 124. Kd2 {+0.01/4 104s} Re4 {0.00/42 51s} 125. e6 {+0.01/3 18s} Kh3 {0.00/42 74s} 126. Kd3 {+0.01/3 52s} Re1 {0.00/42 57s} 127. Rh6+ {+0.01/3 91s} Kg4 {0.00/43 86s} 128. Kd2 {+0.01/3 59s} Re4 {0.00/42 53s} 129. Kd3 {+0.01/3 45s} Re1 {0.00/40 49s} 130. Kd2 {+0.01/3 67s} Re4 {0.00/39 53s} 131. Rg6+ {0.00/1002 70s} Kf5 {0.00/39 50s} 132. Rxg3 {+0.09/3 40s} Rxd4+ {0.00/42 52s} 133. Kc3 {+0.05/4 34s} Re4 {0.00/44 56s} 134. e7 {+0.03/3 34s} Rxe7 {0.00/39 26s} 135. Kc4 {+0.02/3 93s} Re4+ {0.00/42 51s} 136. Kc5 {+0.05/4 61s} Re5+ {0.00/46 52s} 137. Kxc6 {+0.04/4 38s} Rb5 {0.00/50 52s} 138. Rf3+ {+0.02/4 131s} Ke4 {0.00/60 57s} 139. Ra3 {+0.04/3 19s} Rxb4 {0.00/55 29s} 140. Rxa6 {+0.04/3 31s} Kd3 {0.00/58 57s} 141. Ra8 {+0.03/4 135s} Kc3 {0.00/59 58s, Draw by adjudication: user decision} 1/2-1/2[/pgn]

=================================================

But if the opponents will prepare some very sharp openings, the things can turn different for Leela. From "Fried Liver attack", the first game was a quick win for Komodo:

[pgn][Event "My Tournament"] [Site "?"] [Date "2018.07.15"] [Round "1"] [White "Komodo 12.1.1"] [Black "lc0 ID491"] [Result "1-0"] [FEN "r2k1b1r/pp4pp/n2N2q1/3pPb2/8/P4P2/1PP1Q1PP/R1B2RK1 w - - 2 1"] [GameDuration "01:22:34"] [GameEndTime "2018-07-15T09:21:01.961 GTB Daylight Time"] [GameStartTime "2018-07-15T07:58:27.805 GTB Daylight Time"] [PlyCount "39"] [SetUp "1"] [TimeControl "3600+60"] 1. Bf4 {+0.72/28 199s} Rc8 {-0.27/3 227s} 2. Nxb7+ {+0.59/31 254s} Ke7 {-0.37/3 190s} 3. e6 {+0.70/30 105s} Qxe6 {-0.67/3 175s} 4. Bd6+ {+0.61/32 154s} Kd7 {-0.70/3 92s} 5. Qb5+ {+0.84/31 132s} Rc6 {-0.70/4 0s} 6. Rfe1 {+0.80/32 158s} Qg6 {-1.10/3 267s} 7. Bxf8 {+0.74/33 147s} Rxf8 {-1.16/3 95s} 8. Na5 {+0.68/34 219s} Rb8 {-0.89/3 301s} 9. Re7+ {+3.12/29 131s} Kd6 {-1.30/4 184s} 10. Rb7 {+4.58/26 114s} Rbc8 {-2.88/3 202s} 11. Rd1 {+6.67/23 163s} Qe6 {-2.89/3 109s} 12. c4 {+7.58/26 126s} Bd3 {-4.29/3 192s} 13. Rxa7 {+8.24/27 196s} Bxc4 {-4.59/3 212s} 14. Nxc4+ {+M13/31 12s} Rxc4 {-4.62/4 0.044s} 15. Rxd5+ {+M11/43 12s} Qxd5 {-1.40/3 0.016s} 16. Rd7+ {+M9/99 1.4s} Ke6 {-51.72/4 195s} 17. Qxd5+ {+M7/99 0.014s} Kf6 {-55.41/4 0s} 18. Qf7+ {+M5/99 0.006s} Kg5 {-64.93/4 153s} 19. Rd5+ {+M3/99 0.005s} Kh4 {-82.11/1002 237s} 20. Qh5# {+M1/99 0.005s, White mates} 1-0[/pgn]

In the second game, Komodo, now as black, again gained some advantage in the endgame, but it ended in a draw.

[pgn][Event "My Tournament"] [Site "?"] [Date "2018.07.15"] [Round "2"] [White "lc0 ID491"] [Black "Komodo 12.1.1"] [Result "1/2-1/2"] [FEN "r2k1b1r/pp4pp/n2N2q1/3pPb2/8/P4P2/1PP1Q1PP/R1B2RK1 w - - 2 1"] [GameDuration "03:52:41"] [GameEndTime "2018-07-15T13:13:43.525 GTB Daylight Time"] [GameStartTime "2018-07-15T09:21:01.992 GTB Daylight Time"] [PlyCount "125"] [SetUp "1"] [Termination "adjudication"] [TimeControl "3600+60"] 1. Bg5+ {+0.38/3 193s} Kc7 {-0.10/29 186s} 2. Qd2 {+0.55/3 162s} Kb8 {-0.42/30 233s} 3. Rad1 {+0.59/3 100s} Nc7 {-0.27/29 195s} 4. c4 {+0.66/3 92s} d4 {0.00/30 115s} 5. Qxd4 {+0.66/3 72s} Qxg5 {0.00/30 64s} 6. Nf7 {+0.61/3 85s} Ne6 {0.00/34 121s} 7. Qd7 {+0.58/3 17s} Bc5+ {0.00/34 152s} 8. Kh1 {+0.76/4 196s} Qxg2+ {0.00/36 142s} 9. Kxg2 {+2.90/3 0.035s} Nf4+ {0.00/35 129s} 10. Kg3 {+0.74/4 201s} Ne2+ {0.00/35 130s} 11. Kh4 {+0.79/4 103s} Bxd7 {0.00/30 70s} 12. Rxd7 {+0.82/3 65s} Rc8 {0.00/32 122s} 13. b4 {+0.88/3 199s} Bf8 {0.00/33 117s} 14. Re1 {+0.79/3 107s} Rxc4+ {0.00/28 49s} 15. Kg5 {-0.26/3 203s} h6+ {0.00/33 141s} 16. Kg6 {-0.35/4 185s} Nf4+ {0.00/34 95s} 17. Kh7 {+0.10/4 300s} Rc7 {0.00/35 93s} 18. Rd8+ {+0.03/3 143s} Rc8 {0.00/5 0s} 19. Rxc8+ {+0.02/3 111s} Kxc8 {0.00/5 0s} 20. e6 {+0.02/3 63s} Be7 {0.00/36 126s} 21. Kxg7 {+0.01/3 58s} Kc7 {0.00/36 97s} 22. Nxh6 {+0.01/3 0.26s} Rf8 {0.00/36 101s} 23. Nf7 {0.00/3 0.26s} Re8 {0.00/37 160s} 24. Re4 {0.00/3 0.25s} Nd5 {0.00/38 153s} 25. Kg6 {0.00/3 0.24s} Bd6 {0.00/36 99s} 26. Rc4+ {0.00/3 0.39s} Kb6 {0.00/39 98s} 27. Nxd6 {0.00/3 75s} Rxe6+ {0.00/41 152s} 28. Kg5 {0.00/4 50s} Rxd6 {+0.07/33 108s} 29. h4 {0.00/3 0.61s} Rd8 {+0.07/34 108s} 30. h5 {-0.04/3 221s} Rg8+ {+0.25/33 113s} 31. Kh4 {-0.05/3 120s} Ne3 {+0.07/34 114s} 32. Rc5 {-0.05/3 69s} Ng2+ {+0.07/35 125s} 33. Kh3 {-0.05/4 0.042s} Nf4+ {+0.07/35 112s} 34. Kh4 {-0.06/3 0.67s} Ne6 {+0.07/37 107s} 35. Rc4 {-0.06/3 27s} Rg1 {+0.07/37 111s} 36. f4 {-0.06/3 0.43s} Nf8 {+0.17/34 144s} 37. Re4 {+0.03/3 246s} Kc7 {+0.22/34 92s} 38. Re7+ {-0.05/3 250s} Nd7 {+0.26/36 200s} 39. Rh7 {-0.17/3 390s} Kc6 {+0.36/32 120s} 40. h6 {+0.19/3 216s} Kd6 {+0.38/35 88s} 41. a4 {-0.45/4 247s} Rf1 {+0.37/33 92s} 42. Kg3 {-0.61/3 383s} Rh1 {+0.37/36 117s} 43. f5 {-0.69/3 80s} a6 {+0.38/37 83s} 44. Kf4 {-0.71/3 30s} Rh5 {+0.38/37 89s} 45. Rh8 {-0.92/3 231s} Nf6 {+0.38/37 120s} 46. Rb8 {-0.92/3 75s} Nd5+ {+0.39/36 91s} 47. Kg4 {-0.92/4 34s} Rxh6 {+0.40/37 95s} 48. Rxb7 {-0.93/3 23s} Ke5 {+0.43/37 119s} 49. b5 {-0.94/3 246s} Rf6 {+0.45/36 75s} 50. Ra7 {-0.97/3 354s} axb5 {+0.49/36 90s} 51. axb5 {-0.98/3 89s} Rxf5 {+0.50/37 69s} 52. b6 {-0.92/3 169s} Rf1 {+0.51/41 91s} 53. b7 {-0.93/3 125s} Nf6+ {+0.51/42 89s} 54. Kg3 {-0.94/4 302s} Ne4+ {+0.51/43 106s} 55. Kg4 {-0.88/4 75s} Rg1+ {+0.51/41 80s} 56. Kf3 {-0.91/4 64s} Rg3+ {+0.51/41 76s} 57. Ke2 {-0.92/5 0s} Rb3 {+0.51/42 87s} 58. Ra5+ {-0.98/3 155s} Kd4 {+0.51/42 85s} 59. Ra8 {-0.98/3 43s} Nc3+ {+0.51/42 78s} 60. Kf3 {-0.98/5 64s} Rxb7 {+0.51/44 75s} 61. Kf4 {-0.98/4 55s} Rf7+ {+0.50/39 71s} 62. Kg5 {-1.01/4 92s} Rf1 {+0.50/38 66s} 63. Kg4 {-0.98/3 27s, Draw by adjudication: user decision} 1/2-1/2[/pgn]

Laskos wrote: ↑Sun Jul 15, 2018 10:28 pm A taste of what could happen in the next days at WCCC. 4 LTC games, side and reversed from 2 starting positions. According to my estimates, at this long time control, 60 minutes + 60 seconds increment, lc0 ID491 on GTX 1060 GPU should be a bit stronger than the latest Komodo 12.1.1 on one i7 core from regular, quiet starting positions.

Then the fascinating question will be what happens when we increase the core count of Komodo (do we know what they have in the WCCC?) and the graphics cards of Leela (8x V100 ?)

Werewolf wrote: ↑Sun Jul 15, 2018 10:32 pm

Laskos wrote: ↑Sun Jul 15, 2018 10:28 pm A taste of what could happen in the next days at WCCC. 4 LTC games, side and reversed from 2 starting positions. According to my estimates, at this long time control, 60 minutes + 60 seconds increment, lc0 ID491 on GTX 1060 GPU should be a bit stronger than the latest Komodo 12.1.1 on one i7 core from regular, quiet starting positions.

Then the fascinating question will be what happens when we increase the core count of Komodo (do we know what they have in the WCCC?) and the graphics cards of Leela (8x V100 ?)

I estimated that the improvement from one core to 48 for Komodo is a bit larger than the improvement from my GTX 1060 to 8 x V100. Also, time control in WCCC is larger, but the scaling of Leela at these LTC is hard to know. So, my little experiment is somehow representative, but it's easy to miss 50-100 Elo points up or down. It seems that if the opening is quiet, expect maybe a draw between Leela and Komodo, if sharp, good chances for Komodo to win. Let's see.

A0 on a larger net was +80 elo over SF8 at ~800K nodes/move & +100 elo at ~4M+ npm (taking the 100 game set as absolute). but recall this was SF8 on a 64(?) core machine with 1GB hash. don't know if tests have been done on how that effects scaling, but i assume it's not good.

in terms of raw elo est. +300 elo gained 8K to 80K, and about the same amount from ~20K to 1M. basically by 500K nodes searched/move there were perhaps 50 elo left on the table, but this is all presumably benchmarked against the same flawed stockfish setup.

is +40 elo (perhaps) reasonable to expect for SF going from 70Mnps to 700M?

looking at the chart now i don't know how we're surprised by results like kai's & this test https://docs.google.com/spreadsheets/d/ ... =583567367