Leela (lack of) endgame technique?

[whereagles]

Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves

Case in point:
https://www.chess.com/computer-chess-championship
game 410

[Jouni]

Yes blog says: "She had 4 chances to win in this endgame and along with the other 2 chances to win easily in late middlegame earlier, she lost 6 times the move to offer the finishing blow." Also I read somewhere, that Leela cannot convert KBNK 100%. Idea: switch to Stockfish or smilar trad. engine in endgame!

[Joerg Oster]

Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves

Case in point:
https://www.chess.com/computer-chess-championship
game 410

I'm quite skeptical about doing no rollouts also in late endgame.
The search needs to choose a move (out of many winning moves) that leads to a sure win.

I strongly doubt the NN can learn this for all kind of possible, and sometimes very difficult to win, endgame situations.

[Spliffjiffer]

LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases

[AdminX]

LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases

I wish Robert Hyatt were still active on Talkchess, I would love to hear his take on LC0.

[Werewolf]

I wish Robert Hyatt were still active on Talkchess, I would love to hear his take on LC0.

The Changing of the Guard. Robert retired just before Lc0 came along, like they were never meant to meet each other.

[ernest]

The Changing of the Guard. Robert retired just before Lc0 came along, like they were never meant to meet each other.

Come on, Bob is still a Moderator of this Forum, as far as I know, so why couldn't he give us his opinion ?

[Uri Blass]

LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases

I think that the main reason that it took a long time for a-b engines to get 3300 elo is because people did not have a good hardware.
Good hardware help also to test faster so it is not fair to compare times that it took a-b engines to get 3300 and time that it took LC0 to get 3300.

I also think that LC0 use a significantly superior hardware in the CCCC and GPU today is simply superior to CPU for tasks like chess.

I wonder if it was the case of the past
What is the rating gap between stockfish and LC0 if you can use only hardware of 2010(CPU or GPU)?

Edit:Note that for me better hardware means better hardware in what you can do with it.
I do not care about the price or performance per watt but about results that is possible to achieve for the specific problem.

For example
if you can find all the prime below 10^10 in 10 seconds with hardware A and you need 100 seconds with hardware B then hardware A is better for the specific problem.

[Spliffjiffer]

yes, sure, you are right that hardware-developement plays a major role in computerchess and the speed of progress as well !
OTOH: if u state that after around 3-4 months of training a NN gets a playing-strengh that is lets say 200 elo weaker than the top of the crop (a-b-engines (PC) after more than 25 years of developement) then its obvious to believe that this new approach made quicker progress than alpha-beta-prunning did and will substitute or merge with the "old" approach, isnt it?...oc we can speculate that there is a "wall" where NN's in chess stagnate and the best approach stays the a-b-prunning approach but only time can show us...i dont believe in it but i err too often to be sure
imo we witness the end of PURE a-b-engines within the next few years...just a personal point of view in respect of the data i get from the recent developement of NN's+chess and their meassured strengh.

[Nay Lin Tun]

Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves

Case in point:
https://www.chess.com/computer-chess-championship
game 410

Be aware that the version of Leela playing over there has
1. No TB support
2. Not the best version ( best version of 10xxx net is about +25 elo better)
3. 50 moves rules bug in almost whole training
4. Terrible Time managment with pondering on setting (not properly tested)

These conditions cause leela about -100 elo.

And 20xxxx are training on different methods.

Watch out new upcoming leela in next couple of weeks!
Btw, new 20xxxx nets are close to 3000 rating now!!