Leela (lack of) endgame technique?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
whereagles
Posts: 536
Joined: Thu Nov 13, 2014 11:03 am

Leela (lack of) endgame technique?

Post by whereagles » Thu Sep 13, 2018 7:45 pm

Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves :)

Case in point:
https://www.chess.com/computer-chess-championship
game 410

Jouni
Posts: 1854
Joined: Wed Mar 08, 2006 7:15 pm

Re: Leela (lack of) endgame technique?

Post by Jouni » Thu Sep 13, 2018 8:01 pm

Yes blog says: "She had 4 chances to win in this endgame and along with the other 2 chances to win easily in late middlegame earlier, she lost 6 times the move to offer the finishing blow." Also I read somewhere, that Leela cannot convert KBNK 100%. Idea: switch to Stockfish or smilar trad. engine in endgame!
Jouni

Joerg Oster
Posts: 640
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Leela (lack of) endgame technique?

Post by Joerg Oster » Thu Sep 13, 2018 8:30 pm

whereagles wrote:
Thu Sep 13, 2018 7:45 pm
Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves :)

Case in point:
https://www.chess.com/computer-chess-championship
game 410
I'm quite skeptical about doing no rollouts also in late endgame.
The search needs to choose a move (out of many winning moves) that leads to a sure win.

I strongly doubt the NN can learn this for all kind of possible, and sometimes very difficult to win, endgame situations.
Jörg Oster

Spliffjiffer
Posts: 145
Joined: Thu Aug 02, 2012 5:48 pm
Location: Germany

Re: Leela (lack of) endgame technique?

Post by Spliffjiffer » Thu Sep 13, 2018 9:13 pm

LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases ;-)
Wahrheiten sind Illusionen von denen wir aber vergessen haben dass sie welche sind.

User avatar
AdminX
Posts: 4934
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: Leela (lack of) endgame technique?

Post by AdminX » Thu Sep 13, 2018 9:20 pm

Spliffjiffer wrote:
Thu Sep 13, 2018 9:13 pm
LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases ;-)
I wish Robert Hyatt were still active on Talkchess, I would love to hear his take on LC0.
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Werewolf
Posts: 1061
Joined: Thu Sep 18, 2008 8:24 pm

Re: Leela (lack of) endgame technique?

Post by Werewolf » Thu Sep 13, 2018 9:33 pm

AdminX wrote:
Thu Sep 13, 2018 9:20 pm

I wish Robert Hyatt were still active on Talkchess, I would love to hear his take on LC0.
The Changing of the Guard. Robert retired just before Lc0 came along, like they were never meant to meet each other.

ernest
Posts: 1796
Joined: Wed Mar 08, 2006 7:30 pm

Re: Leela (lack of) endgame technique?

Post by ernest » Fri Sep 14, 2018 1:34 am

Werewolf wrote:
Thu Sep 13, 2018 9:33 pm
The Changing of the Guard. Robert retired just before Lc0 came along, like they were never meant to meet each other.
Come on, Bob is still a Moderator of this Forum, as far as I know, so why couldn't he give us his opinion ?

Uri Blass
Posts: 8217
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Leela (lack of) endgame technique?

Post by Uri Blass » Fri Sep 14, 2018 4:54 am

Spliffjiffer wrote:
Thu Sep 13, 2018 9:13 pm
LC0 (formerly LCZero), a new approach in computerchess to play the best chess by using a "Neural Network" instead of prunning moves away by "alpha-beta-prunning" (as current engines are based on still momentarely as SF/Houdini/Komodo are), is more or less 1 year among us and performs increadable ca 3300 elo @ CCCC1...
can u guys remember how long it took us to make the a-b-prunning approach working that well that the engines were considered GM-strengh ?
it took many....., many years....
now, with this (absolutely) amazing new concept, this "thing" (no influence by human being...it just plays the game vs itself again and again and noone has ever sayed a single thing about chess exept the rules) plays chess already (lets say: it trained chess for around 3-4 months and now is getting restarted and restarted for reasons i dont have a look for) on a level that NO human is competitive with (probably!?...still no competition was brought into life against a GM)...if u like to have a chess-engine from scratch that is superior to any contender and is build within a few months then leave solarsystem, diss the programmers and wait until it falls into your lap while trolling about their inaccuracies... but for real (imo):
in a few years at the latest i think this will be the new monument of chessprogramming (how far away from the top seats is LC0 currently, 200 elo ? lol)
does it lack of knowledge?...yes oc as a-b-engines...will it lack of knowledge in the future?...yes oc as a-b-engines(in my opinion they will stay important for tactical discourses)....will this approach overshadow the former programmers approach of alpha-beta-cutting...yes but thats normal with more efficient ways to walk...will it play ever perfect chess?...yes oc, with 32 men tablebases ;-)

I think that the main reason that it took a long time for a-b engines to get 3300 elo is because people did not have a good hardware.
Good hardware help also to test faster so it is not fair to compare times that it took a-b engines to get 3300 and time that it took LC0 to get 3300.

I also think that LC0 use a significantly superior hardware in the CCCC and GPU today is simply superior to CPU for tasks like chess.

I wonder if it was the case of the past
What is the rating gap between stockfish and LC0 if you can use only hardware of 2010(CPU or GPU)?

Edit:Note that for me better hardware means better hardware in what you can do with it.
I do not care about the price or performance per watt but about results that is possible to achieve for the specific problem.

For example
if you can find all the prime below 10^10 in 10 seconds with hardware A and you need 100 seconds with hardware B then hardware A is better for the specific problem.

Spliffjiffer
Posts: 145
Joined: Thu Aug 02, 2012 5:48 pm
Location: Germany

Re: Leela (lack of) endgame technique?

Post by Spliffjiffer » Fri Sep 14, 2018 1:35 pm

yes, sure, you are right that hardware-developement plays a major role in computerchess and the speed of progress as well !
OTOH: if u state that after around 3-4 months of training a NN gets a playing-strengh that is lets say 200 elo weaker than the top of the crop (a-b-engines (PC) after more than 25 years of developement) then its obvious to believe that this new approach made quicker progress than alpha-beta-prunning did and will substitute or merge with the "old" approach, isnt it?...oc we can speculate that there is a "wall" where NN's in chess stagnate and the best approach stays the a-b-prunning approach but only time can show us...i dont believe in it but i err too often to be sure :-)
imo we witness the end of PURE a-b-engines within the next few years...just a personal point of view in respect of the data i get from the recent developement of NN's+chess and their meassured strengh.
Wahrheiten sind Illusionen von denen wir aber vergessen haben dass sie welche sind.

Nay Lin Tun
Posts: 343
Joined: Mon Jan 16, 2012 5:34 am

Re: Leela (lack of) endgame technique?

Post by Nay Lin Tun » Fri Sep 14, 2018 2:38 pm

whereagles wrote:
Thu Sep 13, 2018 7:45 pm
Hi all,

What's up with leela's seemingly inability to win endgames? At CCCC it outplayed komodo in the middle game but didn't manage to convert what K evaluated as "clear win".

I know NN have trouble selecting moves when many lines seem to win, but it should at least choose actually winning moves :)

Case in point:
https://www.chess.com/computer-chess-championship
game 410
Be aware that the version of Leela playing over there has
1. No TB support
2. Not the best version ( best version of 10xxx net is about +25 elo better)
3. 50 moves rules bug in almost whole training
4. Terrible Time managment with pondering on setting (not properly tested)

These conditions cause leela about -100 elo.

And 20xxxx are training on different methods.

Watch out new upcoming leela in next couple of weeks!
Btw, new 20xxxx nets are close to 3000 rating now!!

:o :shock: :roll: :lol: 8-) :D

Post Reply