Leela (lack of) endgame technique?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
corres
Posts: 3336
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Leela (lack of) endgame technique?

Post by corres » Fri Sep 14, 2018 3:14 pm

Nay Lin Tun wrote:
Fri Sep 14, 2018 2:38 pm
Be aware that the version of Leela playing over there has
1. No TB support
2. Not the best version ( best version of 10xxx net is about +25 elo better)
3. 50 moves rules bug in almost whole training
4. Terrible Time managment with pondering on setting (not properly tested)
These conditions cause leela about -100 elo.
...
...and engines do not use Opening Book (as it was AlphaZero-Stockfish 8 match), TC is rapid
(15min + 5sec/move), Stockfish use 8 GB memory but 4 X Tesla V100 use 64 GB memory -
so these circumstances give Leela ~200 Elo benefit...
At the end of Stage 2 we may take a little realer picture about the power of Leela.

corres
Posts: 3336
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Leela (lack of) endgame technique?

Post by corres » Fri Sep 14, 2018 3:52 pm

Leela Chess Zero - like AlphaZero - has no any "technique".
Chess engines based on Neural Network have only a memory with big capacity and a searching motor (MCTS)
to find appropriate move memorized during learning cycle for the analyzed position.
Physically the measure of NN memory is not big (some 100MB) but using filters make the memory virtually
more bigger.
If developers of Leela want to be Leela with good endgame knowledge they ought to growth the measure of NN and to play much more learning play.

Uri Blass
Posts: 8787
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Leela (lack of) endgame technique?

Post by Uri Blass » Sat Sep 15, 2018 7:00 am

Spliffjiffer wrote:
Fri Sep 14, 2018 1:35 pm
yes, sure, you are right that hardware-developement plays a major role in computerchess and the speed of progress as well !
OTOH: if u state that after around 3-4 months of training a NN gets a playing-strengh that is lets say 200 elo weaker than the top of the crop (a-b-engines (PC) after more than 25 years of developement) then its obvious to believe that this new approach made quicker progress than alpha-beta-prunning did and will substitute or merge with the "old" approach, isnt it?...oc we can speculate that there is a "wall" where NN's in chess stagnate and the best approach stays the a-b-prunning approach but only time can show us...i dont believe in it but i err too often to be sure :-)
imo we witness the end of PURE a-b-engines within the next few years...just a personal point of view in respect of the data i get from the recent developement of NN's+chess and their meassured strengh.
I do not agree that after 3-4 months of training a NN gets playing strength that is 200 elo weaker than the top of a-b engines.
I think the NN simply use a significantly superior hardware.

The only fair comparison is to give the NN the same hardware (CPU and not GPU) because GPU today is significantly superior hardware for chess.

I believe that a-b engines can be significantly stronger if they use the GPU and not the CPU but they are not optimized to use the GPU today.

If I understand correctly stockfish today cannot even use the GPU because it is not programmed to do it so we cannot check what is stockfish's initial rating using the GPU.

Note that I expect worse initial rating for stockfish in case that somebody convert it to use the GPU but if the same person continue to develop stockfish GPU then I expect stockfish GPU to become better than Stockfish CPU after enough time of developement.

The point is that there may be changes in the algorithm that are productive with GPU and counterproductive with CPU(for example maybe it is possible to have some better evaluation function that is too expensive for the CPU but cheap with the GPU).

Spliffjiffer
Posts: 248
Joined: Thu Aug 02, 2012 5:48 pm
Location: Germany

Re: Leela (lack of) endgame technique?

Post by Spliffjiffer » Sat Sep 15, 2018 12:02 pm

hi Uri, maybe maybe future will show us :-)
in terms of "fair comparison" though its much too personally to get ever consensus
sounds like Compare apples to pears to me ;-)
Wahrheiten sind Illusionen von denen wir aber vergessen haben dass sie welche sind.

muxecoid
Posts: 150
Joined: Sat Jan 30, 2010 9:54 am
Location: Israel

Re: Leela (lack of) endgame technique?

Post by muxecoid » Sun Sep 16, 2018 1:25 pm

The last CCCC game against Pedone was disgusting. Any proper modern engine would win the endgame.

MikeGL
Posts: 953
Joined: Thu Sep 01, 2011 12:49 pm

Re: Leela (lack of) endgame technique?

Post by MikeGL » Sun Sep 16, 2018 3:09 pm

I just checked with SF and the endgame phase is just around +1.xx for Lc0.
Endgame Tablebase said that position is drawn, as strange as it may seem.

After 81...Rxe7 of black, this position is a TB draw.
No way for Lc0 to win this, even if it is a pawn up with a major piece vs. a minor piece.

[d]8/4r3/8/3B4/1k6/p7/K7/8 w - - 0 82

muxecoid wrote:
Sun Sep 16, 2018 1:25 pm
The last CCCC game against Pedone was disgusting. Any proper modern engine would win the endgame.
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

muxecoid
Posts: 150
Joined: Sat Jan 30, 2010 9:54 am
Location: Israel

Re: Leela (lack of) endgame technique?

Post by muxecoid » Sun Sep 16, 2018 8:31 pm

I mean earlier position. With rook and minor piece vs two minor pieces. If only exchanging minor pieces was delayed there should have been a way to win.

OliverBr
Posts: 475
Joined: Tue Dec 18, 2007 8:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch
Contact:

Re: Leela (lack of) endgame technique?

Post by OliverBr » Thu May 07, 2020 11:00 pm

Hello together,
I (author of OliThink), I am very impressed about Leela Chess Zero.
It actually follows the path I was trying to do with Oliver. Create a strong chess engine with any chess knowledge whatever (except the rules, of course).
Fortunately I do now have a fine system, a MacBookPro 2018 15inch, with a powerful eGPU, GTX1080Ti. So (in High Sierra), I can use CUDA to get optimal performance from LC0.
Wit this hardware, Leela Chess Zero (0.24.1) completely and utterly owns and humiliates Stockfish 11 in midgame. Is this a surprise? It's difficult so say, because, honestly, the endgame, kind-a is strange. Whatever winning endgame, LC0 sacrifices pieces for nothing, only when 50-moves rule is threatening, it plays fine and wins. It is kind of funny in some way, as it appears like LC0 is torturing the losing side.

A fine example would be this game that was won since move 40:

Code: Select all

[Event "Computer Chess Game"]
[Site "Olivers-MacBook-2.local"]
[Date "2020.05.08"]
[Round "-"]
[White "Lc0 v0.24.1+git.dirty"]
[Black "Stockfish 11 64"]
[Result "1-0"]
[TimeControl "40/180"]
[Annotator "2. +0.29   1... -0.20"]

1. d4 d5 {-0.20/23 4} 2. c4 {+0.29/9 2.6} e6 {-0.06/24 5} 3. Nf3
{+0.28/11 6} Nf6 {-0.07/22 2.0} 4. g3 {+0.29/10 2.9} Be7 {+0.29/23 5} 5.
Bg2 {+0.29/10 2.5} O-O {+0.34/21 1.7} 6. O-O {+0.30/12 3} dxc4
{+0.42/24 0.3} 7. Qc2 {+0.32/12 3} b5 {+0.29/25 4} 8. a4 {+0.45/14 5} b4
{+0.22/26 0.1} 9. Nfd2 {+0.49/18 7} c6 {+0.00/30 0.1} 10. Nxc4
{+0.48/18 1.0} Qxd4 {+0.28/29 1.6} 11. Rd1 {+0.53/18 2.1} Qc5 {+0.00/30 4}
12. Be3 {+0.55/18 0.2} Qh5 {+0.15/23 2.5} 13. Nbd2 {+0.55/16 3} Ng4
{+0.16/24 2.0} 14. Nf3 {+0.43/15 14} Nxe3 {+0.00/31 0.2} 15. Nxe3
{+0.41/13 0.2} a6 {+0.00/28 1.4} 16. Nc4 {+0.75/12 11} Ra7 {+0.31/26 8} 17.
Rac1 {+0.81/14 4} c5 {+0.12/22 2.0} 18. Nfe5 {+0.82/12 1.9} g6
{+0.00/29 17} 19. h4 {+1.03/10 8} Bb7 {+0.00/28 15} 20. Qd3 {+1.08/14 5}
Bxg2 {+0.00/23 2.8} 21. Kxg2 {+1.05/17 0.4} Rc7 {-0.54/33 18} 22. b3
{+1.29/14 6} Qf5 {-0.59/25 2.0} 23. Qe3 {+1.56/13 5} Nc6 {-0.97/27 0.7} 24.
Nxc6 {+1.60/15 4} Rxc6 {-0.97/30 0.1} 25. Rd7 {+1.63/14 2.3} Bf6
{-1.22/32 17} 26. Rcd1 {+1.74/17 1.2} Bc3 {-1.16/25 1.7} 27. Qf3
{+1.77/12 8} Rfc8 {-1.18/31 0.1} 28. Nd6 {+1.75/13 7} Qxf3+ {-1.40/24 4}
29. Kxf3 {+1.76/14 0.2} Ra8 {-1.34/26 2.6} 30. e3 {+1.88/11 8} Rf8
{-1.47/29 7} 31. Nc4 {+1.94/11 6} Rfc8 {-1.49/31 0.2} 32. g4 {+2.00/11 9}
f5 {-1.57/27 12} 33. g5 {+1.98/13 0.9} e5 {-1.54/24 1.4} 34. Nd6
{+1.99/12 2.8} Rf8 {-1.77/26 5} 35. a5 {+2.08/13 4} e4+ {-2.84/27 12} 36.
Kf4 {+2.16/12 0.6} c4 {-3.06/28 6} 37. Nxc4 {+2.28/13 5} Rf7 {-3.30/26 0.2}
38. h5 {+2.44/15 6} gxh5 {-2.92/26 2.8} 39. Rxf7 {+2.52/15 4} Kxf7
{-2.98/24 2.1} 40. Nd6+ {+2.61/15 2.1} Kg6 {-2.95/26 0.3} 41. Nxf5
{+2.73/15 8} Re6 {-3.47/32 9} 42. Nh4+ {+2.84/15 4} Kf7 {-4.22/35 29} 43.
Rd7+ {+3.02/14 0.2} Re7 {-4.33/36 25} 44. Rxe7+ {+3.12/14 0.2} Kxe7
{-4.35/31 2.4} 45. Kxe4 {+3.20/13 10} Be1 {-4.75/35 5} 46. Kf3 {+3.30/13 8}
Bc3 {-5.37/37 23} 47. Ng2 {+3.60/11 46} Be5 {-5.91/37 0.2} 48. Nf4
{+4.27/9 12} Kf8 {-6.16/30 0.2} 49. Nxh5 {+4.94/8 21} Bc7 {-5.71/28 2.8}
50. Nf6 {+4.79/8 11} Bxa5 {-7.03/29 0.2} 51. Nxh7+ {+5.27/9 5} Kf7
{-7.54/29 1.9} 52. Nf6 {+5.76/8 7} Bb6 {-8.66/30 17} 53. Nd5 {+7.21/7 10}
Bd8 {-7.36/26 5} 54. Nxb4 {+11.54/7 4} Bxg5 {-8.81/30 18} 55. Nxa6
{+15.67/6 7} Bd8 {-9.86/28 1.2} 56. Nb4 {+22.78/6 6} Ke6 {-12.11/26 2.6}
57. Na2 {+21.51/6 4} Ke5 {-6.13/25 1.2} 58. Nc3 {+17.64/6 4} Ba5
{-8.58/27 7} 59. Na4 {+15.10/7 3} Kd5 {-9.92/29 4} 60. Nb2 {+16.73/6 2.7}
Bc7 {-10.91/28 4} 61. Nd3 {+19.40/6 2.6} Bd8 {-9.98/27 0.3} 62. Ne1
{+20.52/6 2.2} Bc7 {-11.79/29 3} 63. Nc2 {+20.61/6 1.9} Bd6 {-9.92/25 2.8}
64. Ne1 {+20.68/6 1.7} Bb8 {-11.85/29 0.2} 65. Ng2 {+21.74/6 1.5} Kc5
{-8.89/25 1.0} 66. Nf4 {+22.17/6 1.3} Kb4 {-5.73/21 0.5} 67. Ke4
{+24.19/5 1.1} Kc5 {-13.67/26 6} 68. Nd3+ {+23.80/6 1.0} Kd6
{-13.68/23 0.2} 69. Nc1 {+29.08/5 0.8} Ke6 {-11.70/25 0.9} 70. Ne2
{+30.67/5 0.7} Kd7 {-12.18/26 2.0} 71. Kd3 {+27.15/5 0.6} Bc7
{-11.18/27 2.5} 72. Ke4 {+27.36/5 0.5} Ba5 {-12.66/24 0.1} 73. f4
{+30.48/5 0.5} Ke7 {-12.75/27 1.1} 74. Ng3 {+26.94/5 0.5} Kd6
{-12.58/26 1.1} 75. Nf1 {+23.33/5 0.5} Kc5 {-5.76/17 0.2} 76. Kd3
{+29.02/5 0.5} Bc7 {-6.69/18 0.2} 77. Ke4 {+22.13/5 0.5} Bd8 {-8.06/22 0.5}
78. Nd2 {+33.62/5 0.5} Bf6 {-8.21/19 0.3} 79. Nf3 {+34.00/5 0.5} Bc3
{-9.82/18 0.2} 80. Kd3 {+25.30/5 0.5} Bg7 {-9.43/16 0.1} 81. Ng5
{+26.01/6 21} Kd6 {-12.85/30 10} 82. Nf3 {+23.32/6 18} Bf8 {-25.76/31 30}
83. Ke4 {+27.18/6 16} Ke6 {-47.72/32 17} 84. Nd4+ {+22.65/7 14} Kf7
{-63.75/37 29} 85. Ne2 {+26.77/7 12} Bc5 {-15.39/26 4} 86. Kd3
{+22.35/6 11} Kg6 {-47.97/38 2.2} 87. e4 {+26.45/6 9} Bf2 {-36.48/27 7} 88.
Nc3 {+27.59/6 8} Bc5 {-64.76/37 10} 89. Kc4 {+33.31/6 7} Be3 {-66.94/36 7}
90. Ne2 {+25.68/6 6} Kh5 {-69.76/33 1.7} 91. Kd5 {+28.08/6 6} Kg6
{-71.24/35 2.1} 92. b4 {+31.99/6 5} Kf7 {-64.23/29 5} 93. b5 {+32.91/6 5}
Ke8 {-1000.26/30 7} 94. f5 {+37.74/6 4} Bb6 {-1000.16/35 0.1} 95. e5
{+42.49/6 4} Bd8 {-1000.14/49 1.3} 96. Nf4 {+45.55/6 3} Kf7
{-1000.15/42 1.3} 97. Nh5 {+44.62/6 2.9} Kf8 {-1000.13/34 2.4} 98. Ng3
{+44.37/6 2.6} Kg7 {-1000.15/30 1.3} 99. Nf1 {+48.66/6 2.3} Kh6
{-58.87/23 1.4} 100. Kd6 {+52.05/6 2.0} Kg5 {-147.92/28 6} 101. f6
{+44.77/6 1.8} Ba5 {-1000.14/30 1.4} 102. Ne3 {+55.69/6 1.7} Bb4+
{-1000.14/28 4} 103. Ke6 {+57.44/6 1.5} Bc5 {-1000.14/29 1.4} 104. Nc4
{+61.88/5 1.3} Kf4 {-1000.12/29 1.5} 105. Nd6 {+58.77/5 1.2} Kg4
{-1000.13/34 1.4} 106. Nb7 {+60.70/5 1.1} Ba7 {-1000.13/36 2.2} 107. b6
{+50.44/5 1.0} Bxb6 {-1000.12/28 0.8} 108. Nd6 {+44.83/6 0.8} Be3
{-1000.13/26 1.3} 109. Nc4 {+27.70/5 0.9} Bh6 {-1000.14/28 2.2} 110. Nd6
{+41.89/5 0.7} Kf4 {-1000.16/35 2.0} 111. Nf5 {+16.03/6 0.7} Bf8
{-1000.17/42 0.7} 112. Ne7 {+22.93/6 0.6} Bh6 {-1000.18/40 1.5} 113. Nc6
{+31.52/6 0.8} Kg5 {-1000.19/34 1.2} 114. Nd4 {+20.10/6 0.8} Kg6
{-1000.17/36 0.6} 115. Nc6 {+13.70/6 0.7} Kg5 {-1000.16/35 0.9} 116. f7
{+17.32/6 0.7} Kg6 {-1000.14/42 1.3} 117. Ke7 {+16.08/6 0.6} Be3
{-1000.13/32 1.0} 118. e6 {+39.78/5 0.9} Bc5+ {-1000.12/38 0.7} 119. Kd7
{+15.81/6 0.8} Kf6 {-1000.14/49 1.5} 120. Nd4 {+7.38/7 0.7} Ba3
{-1000.13/50 2.4} 121. Ke8 {+9.20/8 11} Bd6 {-1000.14/44 4} 122. f8=Q+
{+20.35/8 12} Bxf8 {-1000.13/49 4} 123. Kxf8 {+18.03/9 6} Ke5
{-1000.11/59 0.1} 124. Ke7 {+34.22/8 22} Kd5 {-1000.10/53 4} 125. Kf6
{+47.60/7 10} Kd6 {-1000.10/47 4} 126. Nf5+ {+46.88/7 9} Kc6
{-1000.09/50 4} 127. Kf7 {+55.76/7 8} Kc5 {-1000.09/49 4} 128. e7
{+66.30/7 8} Kb4 {-1000.07/58 0.1} 129. Kg8 {+78.42/6 7} Kc3
{-1000.08/48 4} 130. Nh4 {+73.12/6 6} Kd4 {-1000.09/44 4} 131. Kf7
{+70.39/6 6} Kc3 {-1000.08/52 0.1} 132. Nf3 {+77.00/6 5} Kc4
{-1000.08/49 4} 133. Kg6 {+75.95/6 5} Kc5 {-1000.08/47 5} 134. Kf7
{+56.30/6 5} Kd5 {-1000.08/51 4} 135. e8=R {+91.83/6 4} Kc5 {-1000.11/45 4}
136. Rg8 {+83.00/6 4} Kc4 {-1000.12/46 4} 137. Kg7 {+73.16/6 4} Kb4
{-1000.12/48 4} 138. Re8 {+62.09/6 3} Kb3 {-1000.11/49 0.8} 139. Kh8
{+55.16/6 3} Kc4 {-1000.12/49 4} 140. Re1 {+42.46/6 2.9} Kb4
{-1000.12/52 4} 141. Ne5 {+44.28/6 2.6} Kc5 {-1000.13/51 4} 142. Kg8
{+42.33/6 2.4} Kd5 {-1000.12/53 4} 143. Ra1 {+41.26/6 2.3} Kxe5
{-1000.19/43 2.7} 144. Ra2 {+69.56/6 2.1} Ke4 {-1000.21/55 4} 145. Kg7
{+50.07/6 1.9} Kd3 {-1000.17/59 4} 146. Ra5 {+41.53/6 1.8} Kc4
{-1000.14/58 5} 147. Rf5 {+38.61/6 1.8} Kd3 {-1000.15/60 4} 148. Rf8
{+38.09/6 1.8} Kd4 {-1000.15/61 4} 149. Kf7 {+38.42/6 1.8} Kd3
{-1000.14/67 9} 150. Ra8 {+32.86/6 1.8} Kc3 {-1000.14/65 4} 151. Ke8
{+29.87/6 1.8} Kd4 {-1000.15/61 4} 152. Kd8 {+31.04/6 1.8} Ke4
{-1000.15/62 4} 153. Kc7 {+27.75/6 1.8} Kd4 {-1000.14/72 8} 154. Kb6
{+27.82/6 1.8} Ke5 {-1000.14/64 4} 155. Ka5 {+25.54/6 1.8} Ke4
{-1000.15/56 4} 156. Kb4 {+23.93/6 1.8} Ke3 {-1000.13/60 4} 157. Kc3
{+23.99/6 1.8} Kf4 {-1000.12/69 3} 158. Kc2 {+21.95/6 1.8} Ke5
{-1000.13/52 9} 159. Ra1 {+20.43/6 1.8} Kd5 {-1000.13/62 7} 160. Rh1
{+19.60/6 1.8} Ke5 {-1000.14/61 12} 161. Kb3 {+18.04/6 11} Ke4
{-1000.13/64 5} 162. Ka4 {+17.68/6 10} Kf5 {-1000.14/53 5} 163. Ka5
{+16.00/6 9} Ke4 {-1000.14/61 4} 164. Ka6 {+16.45/6 9} Kf3 {-1000.14/62 4}
165. Kb7 {+15.44/6 8} Kg2 {-1000.14/66 4} 166. Rh6 {+14.48/7 8} Kf3
{-1000.14/56 4} 167. Kb8 {+14.37/7 7} Ke4 {-1000.15/54 4} 168. Rh7
{+14.39/6 7} Kf5 {-1000.15/57 4} 169. Re7 {+13.46/6 6} Kf6 {-1000.15/65 4}
170. Re1 {+13.15/7 6} Kf5 {-1000.14/68 0.2} 171. Ka7 {+12.86/6 6} Kf4
{-1000.13/75 4} 172. Re8 {+12.59/6 5} Kg5 {-1000.13/70 4} 173. Rb8
{+12.18/6 5} Kf5 {-1000.14/63 4} 174. Ka6 {+12.96/6 5} Kg5 {-1000.14/59 4}
175. Rb2 {+12.88/6 4} Kf4 {-1000.15/53 4} 176. Kb5 {+12.57/6 4} Ke5
{-1000.13/59 4} 177. Kc4 {+13.03/6 4} Kf6 {-1000.12/65 0.2} 178. Kd5
{+14.74/6 4} Kf5 {-1000.11/72 0.1} 179. Rb1 {+13.55/6 3} Kf4
{-1000.11/62 4} 180. Kd4 {+10.13/7 3} Kf5 {-1000.11/62 4} 181. Re1
{+7.10/7 2.7} Kg4 {-1000.10/67 1.1} 182. Rf1 {+8.14/7 3} Kg5
{-1000.09/71 0.7} 183. Ke4 {+6.95/7 2.7} Kg6 {-1000.08/70 1.8} 184. Ke5
{+7.15/8 2.8} Kg7 {-1000.07/74 1.6} 185. Kf5 {+14.45/7 2.6} Kf7
{-1000.06/74 4} 186. Re1 {+128.00/4 2.0} Kg7 {-1000.05/107 2.2} 187. Re7+
{+128.00/3 4} Kf8 {-1000.04/245 0.1} 188. Kf6 {+128.00/4 2.5} Kg8
{-1000.03/245 0.1} 189. Re4 {+128.00/4 3} Kf8 {-1000.02/245 0.1} 190. Rd4
{+128.00/4 2.8} Kg8 {-1000.02/245 0.1} 191. Rh4 {+128.00/3 2.8} Kf8
{-1000.01/245 0.1} 192. Rh8# {+128.00/2 1.2}
{Xboard adjudication: Checkmate} 1-0
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink

Dann Corbit
Posts: 11495
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Leela (lack of) endgame technique?

Post by Dann Corbit » Fri May 08, 2020 1:13 am

So happy to see the Olithink author back to talkchess! : :D
LC0 is about the same strength as SF on high end hardware, but on an average machine with a good GPU, LC0 kicks butt.
LC0 is especially good at analyzing quiet positions.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Dann Corbit
Posts: 11495
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Leela (lack of) endgame technique?

Post by Dann Corbit » Fri May 08, 2020 1:14 am

See the TCEC tournament for an example of LC0 verses SF on high end CPUs and GPUs
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply