Leela (lack of) endgame technique?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
yurikvelo
Posts: 710
Joined: Sat Dec 06, 2014 1:53 pm

Re: Leela (lack of) endgame technique?

Post by yurikvelo »

Garry Kasparov vs Deep Blue (Computer) · 1-0
Match (1996), Philadelphia, PA USA, rd 6, Feb-17
https://www.chessgames.com/perl/chessgame?gid=1070879

Code: Select all

SF-dev, D=62
+6,31   21.Bxh7+ Kxh7 22.Ng5+ Kg6 23.Qg4 Nf5 24.Nxe6+ Kh7
+4,94   21.b4 dxc4 22.bxa5 Nd5 23.Re2 c3 24.Bxc3 Nxc3       <---- Kasparov
+2,15   21.c5 Bc7 22.b4 Ng6 23.Bc3 Nf8 24.a4 a6
+2,02   21.Re2 Bc7 22.Qd3 Ng6 23.h5 Nf8 24.h6 g6  

LC0, D=11, 1 MN (591226.pb.gz - a default network file)
+1,28   21.c5 Nf5 22.Re2 Bc7 23.b4 b5
+1,29   21.b4 Bc7 22.c5 a6 23.Re2 Nf5
+1,20   21.Qd3 Nf5 22.c5 Bc7 23.Re2 h5
+1,16   21.Re2 dxc4 22.bxc4 Bb6 23.Rd2 h6
+0,68   21.Qe2 Nf5 22.Rd3 Bc7 23.Kg2 h6
+0,82   21.Kg2 Nf5 22.Bxf5 exf5 23.Ne5 Qe6
+0,70   21.Qc2 Nf5 22.Re2 dxc4 23.bxc4 Ncxd4
+0,80   21.h5 Nf5 22.Bxf5 exf5 23.Ne5 Qd6
+0,50   21.Ng5 h6 22.Bh7+ Kh8 23.b4 dxc4
+0,62   21.Bd3 dxc4 22.bxc4 Bb6 23.c5 Bc7
is it possible to make Lc0 like move 21.Bxh7+
tried manual forward-back with no luck
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Leela (lack of) endgame technique?

Post by corres »

Spliffjiffer wrote: Thu Sep 13, 2018 11:13 pm LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases ;-)
Hosanna for Leela and her developers!
But some critics is allowable, Is not it?
Note
For solving the issue I propose using a random generator to choose among moves with equivalent winning chance.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Leela (lack of) endgame technique?

Post by zullil »

yurikvelo wrote: Fri May 08, 2020 8:52 am
Garry Kasparov vs Deep Blue (Computer) · 1-0
Match (1996), Philadelphia, PA USA, rd 6, Feb-17
https://www.chessgames.com/perl/chessgame?gid=1070879

Code: Select all

SF-dev, D=62
+6,31   21.Bxh7+ Kxh7 22.Ng5+ Kg6 23.Qg4 Nf5 24.Nxe6+ Kh7
+4,94   21.b4 dxc4 22.bxa5 Nd5 23.Re2 c3 24.Bxc3 Nxc3       <---- Kasparov
+2,15   21.c5 Bc7 22.b4 Ng6 23.Bc3 Nf8 24.a4 a6
+2,02   21.Re2 Bc7 22.Qd3 Ng6 23.h5 Nf8 24.h6 g6  

LC0, D=11, 1 MN (591226.pb.gz - a default network file)
+1,28   21.c5 Nf5 22.Re2 Bc7 23.b4 b5
+1,29   21.b4 Bc7 22.c5 a6 23.Re2 Nf5
+1,20   21.Qd3 Nf5 22.c5 Bc7 23.Re2 h5
+1,16   21.Re2 dxc4 22.bxc4 Bb6 23.Rd2 h6
+0,68   21.Qe2 Nf5 22.Rd3 Bc7 23.Kg2 h6
+0,82   21.Kg2 Nf5 22.Bxf5 exf5 23.Ne5 Qe6
+0,70   21.Qc2 Nf5 22.Re2 dxc4 23.bxc4 Ncxd4
+0,80   21.h5 Nf5 22.Bxf5 exf5 23.Ne5 Qd6
+0,50   21.Ng5 h6 22.Bh7+ Kh8 23.b4 dxc4
+0,62   21.Bd3 dxc4 22.bxc4 Bb6 23.c5 Bc7
is it possible to make Lc0 like move 21.Bxh7+
tried manual forward-back with no luck
[pgn][Event "Match"]
[Site "Philadelphia, PA USA"]
[Date "1996.02.17"]
[EventDate "?"]
[Round "6"]
[Result "1-0"]
[White "Garry Kasparov"]
[Black "Deep Blue (Computer)"]
[ECO "D30"]
[WhiteElo "?"]
[BlackElo "?"]
[PlyCount "85"]

1.Nf3 d5 2.d4 c6 3.c4 e6 4.Nbd2 Nf6 5.e3 c5 6.b3 Nc6 7.Bb2
cxd4 8.exd4 Be7 9.Rc1 O-O 10.Bd3 Bd7 11.O-O Nh5 12.Re1 Nf4
13.Bb1 Bd6 14.g3 Ng6 15.Ne5 Rc8 16.Nxd7 Qxd7 17.Nf3 Bb4 18.Re3
Rfd8 19.h4 Nge7 20.a3 Ba5 21.b4 Bc7 22.c5 Re8 23.Qd3 g6 24.Re2
Nf5 25.Bc3 h5 26.b5 Nce7 27.Bd2 Kg7 28.a4 Ra8 29.a5 a6 30.b6
Bb8 31.Bc2 Nc6 32.Ba4 Re7 33.Bc3 Ne5 34.dxe5 Qxa4 35.Nd4 Nxd4
36.Qxd4 Qd7 37.Bd2 Re8 38.Bg5 Rc8 39.Bf6+ Kh7 40.c6 bxc6
41.Qc5 Kh6 42.Rb2 Qb7 43.Rb4 1-0
[/pgn]

[d]2rr2k1/pp1qnppp/2n1p3/b2p4/2PP3P/PP2RNP1/1B3P2/1BRQ2K1 w - - 1 21

Lc0 (network 384x30-t60-3350) finds this line rather quickly:

info depth 12 seldepth 36 time 44348 nodes 408349 score cp 215 wdl 827 97 76 nps 11477 tbhits 0 pv b3b4 a5c7 b4b5 c6a5 b1h7 g8h7 f3g5 h7g6 d1g4 f7f5 h4h5 g6h6 g4h4 e7g8 e3e6 g8f6 c4d5 a5b3 c1e1 d8e8 g5f3 h6h7 h5h6 e8e6 d5e6 d7d5 h6g7 h7g7 h4g5 g7h7 g5f6 d5f3
User avatar
yurikvelo
Posts: 710
Joined: Sat Dec 06, 2014 1:53 pm

Re: Leela (lack of) endgame technique?

Post by yurikvelo »

zullil wrote: Fri May 08, 2020 12:31 pm info depth 12 seldepth 36 time 44348 nodes 408349 score cp 215 wdl 827 97 76 nps 11477 tbhits 0 pv b3b4 a5c7 b4b5 c6a5 b1h7 g8h7 f3g5 h7g6 d1g4 f7f5 h4h5 g6h6 g4h4 e7g8 e3e6 g8f6 c4d5 a5b3 c1e1 d8e8 g5f3 h6h7 h5h6 e8e6 d5e6 d7d5 h6g7 h7g7 h4g5 g7h7 g5f6 d5f3
b4 or c5 lines (also winning lines, as well as Re2, Qd3) are selected from zero-depth.

Can Leela evaluate Bxh7 as good move, even if manually forced?
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Leela (lack of) endgame technique?

Post by zullil »

yurikvelo wrote: Fri May 08, 2020 1:13 pm
zullil wrote: Fri May 08, 2020 12:31 pm info depth 12 seldepth 36 time 44348 nodes 408349 score cp 215 wdl 827 97 76 nps 11477 tbhits 0 pv b3b4 a5c7 b4b5 c6a5 b1h7 g8h7 f3g5 h7g6 d1g4 f7f5 h4h5 g6h6 g4h4 e7g8 e3e6 g8f6 c4d5 a5b3 c1e1 d8e8 g5f3 h6h7 h5h6 e8e6 d5e6 d7d5 h6g7 h7g7 h4g5 g7h7 g5f6 d5f3
b4 or c5 lines (also winning lines, as well as Re2, Qd3) are seleced from zero-depth.

Can Leela evaluate Bxh7 as good move, even if manually forced?
Note that the line I posted includes the sacrifice on h7, which your Lc0 output did not seem to show.
zullil
Posts: 6442
Joined: Tue Jan 09, 2007 12:31 am
Location: PA USA
Full name: Louis Zulli

Re: Leela (lack of) endgame technique?

Post by zullil »

zullil wrote: Fri May 08, 2020 12:31 pm
yurikvelo wrote: Fri May 08, 2020 8:52 am
Garry Kasparov vs Deep Blue (Computer) · 1-0
Match (1996), Philadelphia, PA USA, rd 6, Feb-17
https://www.chessgames.com/perl/chessgame?gid=1070879

Code: Select all

SF-dev, D=62
+6,31   21.Bxh7+ Kxh7 22.Ng5+ Kg6 23.Qg4 Nf5 24.Nxe6+ Kh7
+4,94   21.b4 dxc4 22.bxa5 Nd5 23.Re2 c3 24.Bxc3 Nxc3       <---- Kasparov
+2,15   21.c5 Bc7 22.b4 Ng6 23.Bc3 Nf8 24.a4 a6
+2,02   21.Re2 Bc7 22.Qd3 Ng6 23.h5 Nf8 24.h6 g6  

LC0, D=11, 1 MN (591226.pb.gz - a default network file)
+1,28   21.c5 Nf5 22.Re2 Bc7 23.b4 b5
+1,29   21.b4 Bc7 22.c5 a6 23.Re2 Nf5
+1,20   21.Qd3 Nf5 22.c5 Bc7 23.Re2 h5
+1,16   21.Re2 dxc4 22.bxc4 Bb6 23.Rd2 h6
+0,68   21.Qe2 Nf5 22.Rd3 Bc7 23.Kg2 h6
+0,82   21.Kg2 Nf5 22.Bxf5 exf5 23.Ne5 Qe6
+0,70   21.Qc2 Nf5 22.Re2 dxc4 23.bxc4 Ncxd4
+0,80   21.h5 Nf5 22.Bxf5 exf5 23.Ne5 Qd6
+0,50   21.Ng5 h6 22.Bh7+ Kh8 23.b4 dxc4
+0,62   21.Bd3 dxc4 22.bxc4 Bb6 23.c5 Bc7
is it possible to make Lc0 like move 21.Bxh7+
tried manual forward-back with no luck
[pgn][Event "Match"]
[Site "Philadelphia, PA USA"]
[Date "1996.02.17"]
[EventDate "?"]
[Round "6"]
[Result "1-0"]
[White "Garry Kasparov"]
[Black "Deep Blue (Computer)"]
[ECO "D30"]
[WhiteElo "?"]
[BlackElo "?"]
[PlyCount "85"]

1.Nf3 d5 2.d4 c6 3.c4 e6 4.Nbd2 Nf6 5.e3 c5 6.b3 Nc6 7.Bb2
cxd4 8.exd4 Be7 9.Rc1 O-O 10.Bd3 Bd7 11.O-O Nh5 12.Re1 Nf4
13.Bb1 Bd6 14.g3 Ng6 15.Ne5 Rc8 16.Nxd7 Qxd7 17.Nf3 Bb4 18.Re3
Rfd8 19.h4 Nge7 20.a3 Ba5 21.b4 Bc7 22.c5 Re8 23.Qd3 g6 24.Re2
Nf5 25.Bc3 h5 26.b5 Nce7 27.Bd2 Kg7 28.a4 Ra8 29.a5 a6 30.b6
Bb8 31.Bc2 Nc6 32.Ba4 Re7 33.Bc3 Ne5 34.dxe5 Qxa4 35.Nd4 Nxd4
36.Qxd4 Qd7 37.Bd2 Re8 38.Bg5 Rc8 39.Bf6+ Kh7 40.c6 bxc6
41.Qc5 Kh6 42.Rb2 Qb7 43.Rb4 1-0
[/pgn]

[d]2rr2k1/pp1qnppp/2n1p3/b2p4/2PP3P/PP2RNP1/1B3P2/1BRQ2K1 w - - 1 21

Lc0 (network 384x30-t60-3350) finds this line rather quickly:

info depth 12 seldepth 36 time 44348 nodes 408349 score cp 215 wdl 827 97 76 nps 11477 tbhits 0 pv b3b4 a5c7 b4b5 c6a5 b1h7 g8h7 f3g5 h7g6 d1g4 f7f5 h4h5 g6h6 g4h4 e7g8 e3e6 g8f6 c4d5 a5b3 c1e1 d8e8 g5f3 h6h7 h5h6 e8e6 d5e6 d7d5 h6g7 h7g7 h4g5 g7h7 g5f6 d5f3
Move stats (worst to best order) after 100M nodes of search:

Code: Select all

0508 09:06:59.479934 139740544231168 ../../src/mcts/search.cc:374] === Move stats:
0508 09:06:59.479948 139740544231168 ../../src/mcts/search.cc:375] d1e1  (75  ) N:     466 (+ 0) (P:  0.37%) (WL: -0.90820) (D:  0.030) (M:  3.7) (Q: -0.90820) (U: 2.09135) (S:  1.18315) (V:  -.----) 
0508 09:06:59.479960 139740544231168 ../../src/mcts/search.cc:375] b1f5  (41  ) N:     466 (+ 0) (P:  0.38%) (WL: -0.93340) (D:  0.027) (M:  3.5) (Q: -0.93340) (U: 2.12203) (S:  1.18863) (V:  -.----) 
0508 09:06:59.479966 139740544231168 ../../src/mcts/search.cc:375] b1e4  (39  ) N:     504 (+ 0) (P:  0.39%) (WL: -0.88070) (D:  0.037) (M:  3.9) (Q: -0.88070) (U: 2.05741) (S:  1.17670) (V:  -.----) 
0508 09:06:59.479986 139740544231168 ../../src/mcts/search.cc:375] d1d2  (81  ) N:     537 (+ 0) (P:  0.43%) (WL: -0.90809) (D:  0.030) (M:  3.8) (Q: -0.90809) (U: 2.09191) (S:  1.18382) (V:  -.----) 
0508 09:06:59.479992 139740544231168 ../../src/mcts/search.cc:375] e3e4  (551 ) N:     571 (+ 0) (P:  0.44%) (WL: -0.86920) (D:  0.035) (M:  3.8) (Q: -0.86920) (U: 2.04226) (S:  1.17306) (V:  -.----) 
0508 09:06:59.479998 139740544231168 ../../src/mcts/search.cc:375] b1g6  (43  ) N:     641 (+ 0) (P:  0.52%) (WL: -0.92796) (D:  0.027) (M:  3.4) (Q: -0.92796) (U: 2.11711) (S:  1.18915) (V:  -.----) 
0508 09:06:59.480003 139740544231168 ../../src/mcts/search.cc:375] c1c3  (61  ) N:     673 (+ 0) (P:  0.42%) (WL: -0.55520) (D:  0.160) (M:  6.4) (Q: -0.55520) (U: 1.63848) (S:  1.08328) (V:  -.----) 
0508 09:06:59.480008 139740544231168 ../../src/mcts/search.cc:375] e3e6  (560 ) N:     692 (+ 0) (P:  0.44%) (WL: -0.59532) (D:  0.129) (M:  6.4) (Q: -0.59532) (U: 1.69148) (S:  1.09616) (V:  -.----) 
0508 09:06:59.480013 139740544231168 ../../src/mcts/search.cc:375] e3c3  (544 ) N:     890 (+ 0) (P:  0.49%) (WL: -0.40397) (D:  0.218) (M:  7.2) (Q: -0.40397) (U: 1.44704) (S:  1.04307) (V:  -.----) 
0508 09:06:59.480018 139740544231168 ../../src/mcts/search.cc:375] e3e1  (534 ) N:    1018 (+ 0) (P:  0.54%) (WL: -0.36637) (D:  0.234) (M:  6.2) (Q: -0.36637) (U: 1.39895) (S:  1.03259) (V:  -.----) 
0508 09:06:59.480023 139740544231168 ../../src/mcts/search.cc:375] e3e5  (556 ) N:    1192 (+ 0) (P:  0.53%) (WL: -0.18059) (D:  0.258) (M:  7.1) (Q: -0.18059) (U: 1.15994) (S:  0.97935) (V:  -.----) 
0508 09:06:59.480029 139740544231168 ../../src/mcts/search.cc:375] f3d2  (570 ) N:    1450 (+ 0) (P:  0.63%) (WL: -0.16983) (D:  0.332) (M:  8.9) (Q: -0.16983) (U: 1.14700) (S:  0.97717) (V:  -.----) 
0508 09:06:59.480034 139740544231168 ../../src/mcts/search.cc:375] b2c3  (231 ) N:    1682 (+ 0) (P:  0.53%) (WL:  0.08348) (D:  0.395) (M:  7.8) (Q:  0.08348) (U: 0.82223) (S:  0.90571) (V:  -.----) 
0508 09:06:59.480039 139740544231168 ../../src/mcts/search.cc:375] f3e1  (566 ) N:    1717 (+ 0) (P:  0.67%) (WL: -0.08098) (D:  0.366) (M:  7.2) (Q: -0.08098) (U: 1.03311) (S:  0.95213) (V:  -.----) 
0508 09:06:59.480044 139740544231168 ../../src/mcts/search.cc:375] f3h2  (574 ) N:    1950 (+ 0) (P:  0.76%) (WL: -0.07863) (D:  0.353) (M:  7.2) (Q: -0.07863) (U: 1.02927) (S:  0.95064) (V:  -.----) 
0508 09:06:59.480049 139740544231168 ../../src/mcts/search.cc:375] b1a2  (30  ) N:    2319 (+ 0) (P:  0.64%) (WL:  0.15340) (D:  0.272) (M:  9.5) (Q:  0.15340) (U: 0.73254) (S:  0.88594) (V:  -.----) 
0508 09:06:59.480054 139740544231168 ../../src/mcts/search.cc:375] f3e5  (588 ) N:    2587 (+ 0) (P:  0.86%) (WL:  0.04353) (D:  0.306) (M:  9.5) (Q:  0.04353) (U: 0.87324) (S:  0.91677) (V:  -.----) 
0508 09:06:59.480059 139740544231168 ../../src/mcts/search.cc:375] c1c2  (56  ) N:    2966 (+ 0) (P:  0.86%) (WL:  0.12722) (D:  0.306) (M:  8.7) (Q:  0.12722) (U: 0.76644) (S:  0.89366) (V:  -.----) 
0508 09:06:59.480065 139740544231168 ../../src/mcts/search.cc:375] g3g4  (615 ) N:    3007 (+ 0) (P:  1.39%) (WL: -0.22417) (D:  0.284) (M:  7.6) (Q: -0.22417) (U: 1.21654) (S:  0.99237) (V:  -.----) 
0508 09:06:59.480070 139740544231168 ../../src/mcts/search.cc:375] c4d5  (728 ) N:    3115 (+ 0) (P:  0.97%) (WL:  0.08500) (D:  0.363) (M:  6.6) (Q:  0.08500) (U: 0.82045) (S:  0.90544) (V:  -.----) 
0508 09:06:59.480075 139740544231168 ../../src/mcts/search.cc:375] a3a4  (425 ) N:    4569 (+ 0) (P:  0.84%) (WL:  0.34831) (D:  0.283) (M:  6.9) (Q:  0.34831) (U: 0.48308) (S:  0.83139) (V:  -.----) 
0508 09:06:59.480080 139740544231168 ../../src/mcts/search.cc:375] g1h1  (153 ) N:    5708 (+ 0) (P:  0.83%) (WL:  0.42669) (D:  0.238) (M:  6.6) (Q:  0.42669) (U: 0.38265) (S:  0.80934) (V:  -.----) 
0508 09:06:59.480085 139740544231168 ../../src/mcts/search.cc:375] g1h2  (157 ) N:    7086 (+ 0) (P:  0.99%) (WL:  0.43724) (D:  0.242) (M:  6.6) (Q:  0.43724) (U: 0.36910) (S:  0.80635) (V:  -.----) 
0508 09:06:59.480090 139740544231168 ../../src/mcts/search.cc:375] b2a1  (218 ) N:    7096 (+ 0) (P:  1.11%) (WL:  0.40240) (D:  0.252) (M:  6.9) (Q:  0.40240) (U: 0.41377) (S:  0.81617) (V:  -.----) 
0508 09:06:59.480099 139740544231168 ../../src/mcts/search.cc:375] g1f1  (152 ) N:    7141 (+ 0) (P:  1.03%) (WL:  0.42925) (D:  0.237) (M:  7.0) (Q:  0.42925) (U: 0.37935) (S:  0.80860) (V:  -.----) 
0508 09:06:59.480105 139740544231168 ../../src/mcts/search.cc:375] b1d3  (37  ) N:    8154 (+ 0) (P:  1.25%) (WL:  0.41118) (D:  0.260) (M: 10.1) (Q:  0.41118) (U: 0.40250) (S:  0.81367) (V:  -.----) 
0508 09:06:59.480110 139740544231168 ../../src/mcts/search.cc:375] d1f1  (76  ) N:    8968 (+ 0) (P:  1.30%) (WL:  0.42799) (D:  0.249) (M:  7.5) (Q:  0.42799) (U: 0.38099) (S:  0.80898) (V:  -.----) 
0508 09:06:59.480115 139740544231168 ../../src/mcts/search.cc:375] b1c2  (32  ) N:    8972 (+ 0) (P:  1.32%) (WL:  0.42313) (D:  0.261) (M: 10.2) (Q:  0.42313) (U: 0.38720) (S:  0.81033) (V:  -.----) 
0508 09:06:59.480120 139740544231168 ../../src/mcts/search.cc:375] e3d3  (545 ) N:   13296 (+ 0) (P:  1.26%) (WL:  0.53071) (D:  0.208) (M:  5.5) (Q:  0.53071) (U: 0.24934) (S:  0.78005) (V:  -.----) 
0508 09:06:59.480125 139740544231168 ../../src/mcts/search.cc:375] f3g5  (590 ) N:   37982 (+ 0) (P:  5.64%) (WL:  0.41991) (D:  0.238) (M: 11.9) (Q:  0.41991) (U: 0.39119) (S:  0.81110) (V:  -.----) 
0508 09:06:59.480130 139740544231168 ../../src/mcts/search.cc:375] g1g2  (156 ) N:   68172 (+ 0) (P:  5.30%) (WL:  0.56523) (D:  0.192) (M:  8.6) (Q:  0.56523) (U: 0.20498) (S:  0.77021) (V:  -.----) 
0508 09:06:59.480136 139740544231168 ../../src/mcts/search.cc:375] d1e2  (82  ) N:   91746 (+ 0) (P:  7.14%) (WL:  0.56507) (D:  0.191) (M:  8.2) (Q:  0.56507) (U: 0.20512) (S:  0.77020) (V:  -.----) 
0508 09:06:59.480141 139740544231168 ../../src/mcts/search.cc:375] h4h5  (888 ) N:  119194 (+ 0) (P: 11.36%) (WL:  0.52914) (D:  0.211) (M: 12.8) (Q:  0.52914) (U: 0.25107) (S:  0.78021) (V:  -.----) 
0508 09:06:59.480146 139740544231168 ../../src/mcts/search.cc:375] d1c2  (80  ) N:  137032 (+ 0) (P:  7.12%) (WL:  0.61832) (D:  0.160) (M:  9.1) (Q:  0.61832) (U: 0.13687) (S:  0.75519) (V:  -.----) 
0508 09:06:59.480151 139740544231168 ../../src/mcts/search.cc:375] d1d3  (86  ) N:  398388 (+ 0) (P: 11.22%) (WL:  0.66706) (D:  0.141) (M: 10.7) (Q:  0.66706) (U: 0.07421) (S:  0.74127) (V:  -.----) 
0508 09:06:59.480156 139740544231168 ../../src/mcts/search.cc:375] e3e2  (539 ) N:  990415 (+ 0) (P:  4.91%) (WL:  0.71463) (D:  0.128) (M: 11.0) (Q:  0.71463) (U: 0.01306) (S:  0.72769) (V:  -.----) 
0508 09:06:59.480161 139740544231168 ../../src/mcts/search.cc:375] c4c5  (727 ) N: 5046948 (+ 0) (P: 14.06%) (WL:  0.71769) (D:  0.122) (M: 10.9) (Q:  0.71769) (U: 0.00734) (S:  0.72503) (V:  -.----) 
0508 09:06:59.480166 139740544231168 ../../src/mcts/search.cc:375] b1h7  (45  ) N: 39578859 (+ 0) (P:  2.76%) (WL:  0.71029) (D:  0.076) (M: 20.8) (Q:  0.71029) (U: 0.00018) (S:  0.71047) (V:  -.----) 
0508 09:06:59.480171 139740544231168 ../../src/mcts/search.cc:375] b3b4  (453 ) N: 53432166 (+115) (P:  8.31%) (WL:  0.70477) (D:  0.071) (M: 19.5) (Q:  0.70477) (U: 0.00041) (S:  0.70518) (V:  -.----) 
OliverBr
Posts: 725
Joined: Tue Dec 18, 2007 9:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch

Re: Leela (lack of) endgame technique?

Post by OliverBr »

It took her only 18 moves to crush Komodo 11 with 18. Bxh7. Behold her positional play until this tactical sacrifice.

[pgn][Event "Computer Chess Game"]
[Site "OliversMacBook2.local"]
[Date "2020.05.30"]
[Round "-"]
[White "Lc0 v0.25.1+git.unknown"]
[Black "Komodo 11.01 64-bit"]
[Result "1-0"]
[TimeControl "40/180"]
[Annotator "1. +0.12 1... -0.12"]

1. e4 {+0.12/9} c5 {-0.12/21 9} 2. Nf3 {+0.12/12 0.9} e6 {-0.09/22 12} 3.
Nc3 {+0.17/14 4} a6 {-0.19/23 25} 4. d4 {+0.18/13 0.8} cxd4 {-0.24/22 6} 5.
Nxd4 {+0.19/15 0.2} Qc7 {-0.11/20 12} 6. Bd3 {+0.18/17 4} Nf6 {-0.12/20 5}
7. O-O {+0.17/13 12} Be7 {-0.07/21 5} 8. Qe2 {+0.17/14 1.9} d6 {+0.00/20 7}
9. f4 {+0.14/12 5} O-O {+0.00/21 4} 10. e5 {+0.18/12 7} dxe5 {+0.00/25 0.1}
11. fxe5 {+0.15/14 2.7} Nfd7 {+0.00/24 1.6} 12. Nf3 {+0.11/14 6} Nc6
{+0.13/20 5} 13. Bf4 {+0.09/13 0.2} b5 {+0.30/19 4} 14. Qe1 {+0.14/13 5}
Nb4 {+0.14/20 11} 15. Qg3 {+0.19/15 5} Kh8 {+0.10/21 0.2} 16. Bg5
{+0.17/17 3} Bc5+ {+0.14/21 1.5} 17. Kh1 {+0.14/19 1.4} Bb7 {-0.14/22 10}
18. Bxh7 {+0.49/17 9} Kxh7 {-0.61/22 10} 19. Qh4+ {+0.76/28 0.2} Kg8
{-1.13/20 2.3} 20. Bf6 {+0.74/29 0.2} Rfc8 {-1.35/20 2.8} 21. Bxg7
{+0.81/26 0.7} Kxg7 {-2.65/18 1.5} 22. Ng5 {+0.83/24 1.1} Nf8
{-2.70/19 1.7} 23. Rf6 {+0.79/25 1.4} Be3 {-2.88/20 3} 24. Raf1
{+2.67/20 13} Bxg2+ {-4.21/24 0.2} 25. Kxg2 {+2.66/1 0.2} Ra7
{-4.30/19 2.0} 26. Rxf7+ {+5.34/11 12} Qxf7 {-4.36/26 0.1} 27. Nxf7
{+5.78/11 6} Rxf7 {-4.32/24 0.2} 28. Rxf7+ {+5.35/10 15} Kxf7
{-4.01/19 1.1} 29. Qxb4 {+5.70/8 0.2} Bc1 {-4.20/21 2.1} 30. a4 {+6.57/7 6}
Kg7 {-4.55/22 0.1} 31. axb5 {+8.77/6 5} axb5 {-5.29/22 0.1} 32. Qxb5
{+8.98/6 5} Rc7 {-4.98/21 2.9} 33. Qb3 {+10.29/6 6} Bf4 {-3.67/20 1.7} 34.
Ne2 {+10.78/5 5} Bxe5 {-3.69/19 1.3} 35. c3 {+10.01/6 4} Kf6 {-3.50/20 4}
36. Ng3 {+9.57/6 5} Ng6 {-3.87/20 5} 37. Ne4+ {+8.87/6 4} Kg7 {-3.89/22 4}
38. Kg1 {+10.16/6 5} Re7 {-4.12/21 6} 39. Qb5 {+10.04/5 4} Bf4 {-4.03/19 3}
40. Kf1 {+10.65/5 4} Bxh2 {-3.34/18 1.2} 41. Ke2 {+11.17/6 13} Be5
{-3.91/24 0.1} 42. Kd1 {+12.91/6 12} Bf6 {-4.17/23 2.2} 43. Kc2
{+13.81/6 13} Nf8 {-4.60/23 16} 44. Nxf6 {+17.27/6 11} Kxf6 {-4.39/21 3}
45. Qf1+ {+14.89/6 12} Kg7 {-5.18/23 0.2} 46. b4 {+14.72/6 12} e5
{-5.42/23 0.1} 47. Qg1+ {+11.92/6 10} Kf7 {-5.11/22 6} 48. Qf2+
{+11.94/6 9} Ke8 {-4.65/20 3} 49. Kd2 {+11.75/6 9} Rd7+ {-4.32/21 10} 50.
Kc1 {+11.68/6 8} Rc7 {-3.99/22 6} 51. Kc2 {+12.40/6 7} Ke7 {-4.55/23 9} 52.
Kb3 {+14.74/6 8} Ne6 {-4.91/23 9} 53. Qf5 {+13.65/6 7} Kd6 {-5.27/24 2.5}
54. Qd3+ {+12.45/6 5} Ke7 {-5.47/25 0.2} 55. Qd5 {+13.21/6 6}
{Black Resigns} 1-0
[/pgn]
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink
MOBMAT
Posts: 385
Joined: Sat Feb 04, 2017 11:57 pm
Location: USA

Re: Leela (lack of) endgame technique?

Post by MOBMAT »

[/quote]

I'm quite skeptical about doing no rollouts also in late endgame.
The search needs to choose a move (out of many winning moves) that leads to a sure win.

I strongly doubt the NN can learn this for all kind of possible, and sometimes very difficult to win, endgame situations.
[/quote]

Is Leela using classic MCTS back-propagation or is it using a MCTS solver? A solver should provide better endgame performance.
i7-6700K @ 4.00Ghz 32Gb, Win 10 Home, EGTBs on PCI SSD
Benchmark: Stockfish15.1 NNUE x64 bmi2 (nps): 1277K
User avatar
M ANSARI
Posts: 3707
Joined: Thu Mar 16, 2006 7:10 pm

Re: Leela (lack of) endgame technique?

Post by M ANSARI »

I think it is just the way that NN engines work. Remember they have only been around for a very limited time and for chess it seems that patterns in the opening and in the middle game are much more frequently similar, while in the endgame they need pure brute calculation to the end. Most likely this will disappear as the hardware gets stronger. We might think that a 2080 TI card is a very powerful GPU, but I would equate it to having a 386 CPU for AB engine of a couple of decades ago. There will be a gigantic jump in AI hardware and most likely the "trolling" play that makes Leela seem like an Autistic Savant will fade away with the increase in AI hardware.

There might be another shortcut to this by some new algorithim or new approach to how NN engines work, and that might move things faster. At the moment if you are using engines for chess analysis I think you need a good AB engine for a sanity check.
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Leela (lack of) endgame technique?

Post by Dann Corbit »

OliverBr wrote: Sat May 30, 2020 10:32 pm It took her only 18 moves to crush Komodo 11 with 18. Bxh7. Behold her positional play until this tactical sacrifice.

[pgn][Event "Computer Chess Game"]
[Site "OliversMacBook2.local"]
[Date "2020.05.30"]
[Round "-"]
[White "Lc0 v0.25.1+git.unknown"]
[Black "Komodo 11.01 64-bit"]
[Result "1-0"]
[TimeControl "40/180"]
[Annotator "1. +0.12 1... -0.12"]

1. e4 {+0.12/9} c5 {-0.12/21 9} 2. Nf3 {+0.12/12 0.9} e6 {-0.09/22 12} 3.
Nc3 {+0.17/14 4} a6 {-0.19/23 25} 4. d4 {+0.18/13 0.8} cxd4 {-0.24/22 6} 5.
Nxd4 {+0.19/15 0.2} Qc7 {-0.11/20 12} 6. Bd3 {+0.18/17 4} Nf6 {-0.12/20 5}
7. O-O {+0.17/13 12} Be7 {-0.07/21 5} 8. Qe2 {+0.17/14 1.9} d6 {+0.00/20 7}
9. f4 {+0.14/12 5} O-O {+0.00/21 4} 10. e5 {+0.18/12 7} dxe5 {+0.00/25 0.1}
11. fxe5 {+0.15/14 2.7} Nfd7 {+0.00/24 1.6} 12. Nf3 {+0.11/14 6} Nc6
{+0.13/20 5} 13. Bf4 {+0.09/13 0.2} b5 {+0.30/19 4} 14. Qe1 {+0.14/13 5}
Nb4 {+0.14/20 11} 15. Qg3 {+0.19/15 5} Kh8 {+0.10/21 0.2} 16. Bg5
{+0.17/17 3} Bc5+ {+0.14/21 1.5} 17. Kh1 {+0.14/19 1.4} Bb7 {-0.14/22 10}
18. Bxh7 {+0.49/17 9} Kxh7 {-0.61/22 10} 19. Qh4+ {+0.76/28 0.2} Kg8
{-1.13/20 2.3} 20. Bf6 {+0.74/29 0.2} Rfc8 {-1.35/20 2.8} 21. Bxg7
{+0.81/26 0.7} Kxg7 {-2.65/18 1.5} 22. Ng5 {+0.83/24 1.1} Nf8
{-2.70/19 1.7} 23. Rf6 {+0.79/25 1.4} Be3 {-2.88/20 3} 24. Raf1
{+2.67/20 13} Bxg2+ {-4.21/24 0.2} 25. Kxg2 {+2.66/1 0.2} Ra7
{-4.30/19 2.0} 26. Rxf7+ {+5.34/11 12} Qxf7 {-4.36/26 0.1} 27. Nxf7
{+5.78/11 6} Rxf7 {-4.32/24 0.2} 28. Rxf7+ {+5.35/10 15} Kxf7
{-4.01/19 1.1} 29. Qxb4 {+5.70/8 0.2} Bc1 {-4.20/21 2.1} 30. a4 {+6.57/7 6}
Kg7 {-4.55/22 0.1} 31. axb5 {+8.77/6 5} axb5 {-5.29/22 0.1} 32. Qxb5
{+8.98/6 5} Rc7 {-4.98/21 2.9} 33. Qb3 {+10.29/6 6} Bf4 {-3.67/20 1.7} 34.
Ne2 {+10.78/5 5} Bxe5 {-3.69/19 1.3} 35. c3 {+10.01/6 4} Kf6 {-3.50/20 4}
36. Ng3 {+9.57/6 5} Ng6 {-3.87/20 5} 37. Ne4+ {+8.87/6 4} Kg7 {-3.89/22 4}
38. Kg1 {+10.16/6 5} Re7 {-4.12/21 6} 39. Qb5 {+10.04/5 4} Bf4 {-4.03/19 3}
40. Kf1 {+10.65/5 4} Bxh2 {-3.34/18 1.2} 41. Ke2 {+11.17/6 13} Be5
{-3.91/24 0.1} 42. Kd1 {+12.91/6 12} Bf6 {-4.17/23 2.2} 43. Kc2
{+13.81/6 13} Nf8 {-4.60/23 16} 44. Nxf6 {+17.27/6 11} Kxf6 {-4.39/21 3}
45. Qf1+ {+14.89/6 12} Kg7 {-5.18/23 0.2} 46. b4 {+14.72/6 12} e5
{-5.42/23 0.1} 47. Qg1+ {+11.92/6 10} Kf7 {-5.11/22 6} 48. Qf2+
{+11.94/6 9} Ke8 {-4.65/20 3} 49. Kd2 {+11.75/6 9} Rd7+ {-4.32/21 10} 50.
Kc1 {+11.68/6 8} Rc7 {-3.99/22 6} 51. Kc2 {+12.40/6 7} Ke7 {-4.55/23 9} 52.
Kb3 {+14.74/6 8} Ne6 {-4.91/23 9} 53. Qf5 {+13.65/6 7} Kd6 {-5.27/24 2.5}
54. Qd3+ {+12.45/6 5} Ke7 {-5.47/25 0.2} 55. Qd5 {+13.21/6 6}
{Black Resigns} 1-0
[/pgn]
I think it is positional, not tactical, because of the score at move 18 and because LC0 is an infant, tactically. OK, She'll tear my head off and score a soccer goal with it, but I am comparing her to Stockfish and other Alpha-Beta engines.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.