how will Leela fare at the WCCC?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: how will Leela fare at the WCCC?

Post by jp »

yanquis1972 wrote: Thu Jul 19, 2018 11:50 pm if there was no book, it seems to me leela did awfully well in the opening.
What do you think of as "awfully well"? The Leela guys seem to be making the opposite claim. e.g. "Leela was handicapped by being -3 out of every opening" (not a real quote).
jorose
Posts: 358
Joined: Thu Jan 22, 2015 3:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: how will Leela fare at the WCCC?

Post by jorose »

jdart wrote: Thu Jul 19, 2018 11:45 pm
IanO wrote: Thu Jul 19, 2018 10:41 pmIs this a well-known trap in the Catalan?
No. It looks like 9. Rc1 was Johnny's first move out of book, and that is a very rare move (normal is 9. Rd1 or 9. Bf4). So book knowledge would not have helped Leela here.

13. .. Nh5 looks bad. Stockfish prefers 13. .. g5. And 15. .. f6? was also bad, but Black is gaining material, so it is easy to see how this is tempting.

--Jon
I agree, my impression is that Leela misevaluated the danger to her own king.

I feel like some people in the Leela camp have a tendency to attribute any blunder to tactical weakness and never to evaluation, but if you showed me this game and told me it was two human players I would never have considered saying black made tactical errors. He took a risk pushing pawns in front of his king and underestimated the danger of having a weak king.
-Jonathan
frankp
Posts: 228
Joined: Sun Mar 12, 2006 3:11 pm

Re: how will Leela fare at the WCCC?

Post by frankp »

jorose wrote: Fri Jul 20, 2018 9:36 am
jdart wrote: Thu Jul 19, 2018 11:45 pm
IanO wrote: Thu Jul 19, 2018 10:41 pmIs this a well-known trap in the Catalan?
No. It looks like 9. Rc1 was Johnny's first move out of book, and that is a very rare move (normal is 9. Rd1 or 9. Bf4). So book knowledge would not have helped Leela here.

13. .. Nh5 looks bad. Stockfish prefers 13. .. g5. And 15. .. f6? was also bad, but Black is gaining material, so it is easy to see how this is tempting.

--Jon
I agree, my impression is that Leela misevaluated the danger to her own king.

I feel like some people in the Leela camp have a tendency to attribute any blunder to tactical weakness and never to evaluation, but if you showed me this game and told me it was two human players I would never have considered saying black made tactical errors. He took a risk pushing pawns in front of his king and underestimated the danger of having a weak king.
At the time @crem reported that leela did not even 'see' Bc7.
Whether this is king safety, material or tactics is interesting.
I have no idea how leela 'thinks': for example, whether it evaluates by weighting abstract positional concepts - such as king safety, material, pawn structure etc as a traditional AB search engine is programmed to do. (Still coming to terms with the notion that chess is no more than linear algebra.).

Certainly the king looked draughty to a human eye.
whereagles
Posts: 565
Joined: Thu Nov 13, 2014 12:03 pm

Re: how will Leela fare at the WCCC?

Post by whereagles »

anyway.. disappointing performance. was hoping for a lot more :)
frankp
Posts: 228
Joined: Sun Mar 12, 2006 3:11 pm

Re: how will Leela fare at the WCCC?

Post by frankp »

whereagles wrote: Sat Jul 21, 2018 8:53 am anyway.. disappointing performance. was hoping for a lot more :)
I did not think its performance was inconsistent with its apparent strength indicated by all the 'test' gauntlets etc.
It drew (chess draw) against komodo when running on the better hardware - rather than later when it was on 2 gpu.
But still exhibits its characteristic 'tactical' blindness.
Against such competition on massive hardware, a complete wipe-out would not have been a total surprise. However, it does seem to play chess 'differently' so can often win the occasional game that seems beyond the 'understanding' of even the best ABsearcher engines cf the youtube kingcrusher series.
For that reason alone, I think the project still has great promise of producing something very special - and in one sense it already has: 'true' AI (no one told it how to play) that plays interesting and competent chess.
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: how will Leela fare at the WCCC?

Post by yanquis1972 »

it's alpha or pre-alpha (i'm not a programmer); i fail to see how one loss OTB can be considered truly disappointing.

as frank says, it's going against opening books, massive hardware, & even with its own preposterous HW probably doesn't gain much in elo due to the smaller net. perhaps the missed tactic against jonny would've been spotted, perhaps not, anyone at home willing to take the time can simulate this.

What do you think of as "awfully well"? The Leela guys seem to be making the opposite claim. e.g. "Leela was handicapped by being -3 out of every opening" (not a real quote).
yeah, i didn't get that. that said i haven't analyzed the games in depth & don't have access to (at present) any of the engines. maybe they really like their position out of book, but generally it seemed leela did fine.
crem
Posts: 177
Joined: Wed May 23, 2018 9:29 pm

Re: how will Leela fare at the WCCC?

Post by crem »

jp wrote: Fri Jul 20, 2018 6:47 am
yanquis1972 wrote: Thu Jul 19, 2018 11:50 pm if there was no book, it seems to me leela did awfully well in the opening.
What do you think of as "awfully well"? The Leela guys seem to be making the opposite claim. e.g. "Leela was handicapped by being -3 out of every opening" (not a real quote).
Who said that? That's not really true, Leela does well in opening, and in all WCCC games she went out of openings with both good self-eval and eval from other engines.

Out of entire Lc0 team, I think that an author of the blog is the only person who think that Lc0 would benefit from opening book, so there may be hints to that in the blog.
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: how will Leela fare at the WCCC?

Post by yanquis1972 »

yeah, there definitely were.

unless i really misread, it also said leela was given a small book for at least one game,where it chose a sicilian as black; do you have any insight there? based on what i've seen (at far few nodes) the main net had a pretty good hance of getting slaughtered in most mainlines, don't know if i ever checked out what it does in selfplay against them though.
crem
Posts: 177
Joined: Wed May 23, 2018 9:29 pm

Re: how will Leela fare at the WCCC?

Post by crem »

yanquis1972 wrote: Sat Jul 21, 2018 6:14 pm yeah, there definitely were.

unless i really misread, it also said leela was given a small book for at least one game,where it chose a sicilian as black; do you have any insight there? based on what i've seen (at far few nodes) the main net had a pretty good hance of getting slaughtered in most mainlines, don't know if i ever checked out what it does in selfplay against them though.
It was too small to call it "a book", as it was just one hardcoded move, only to add some variety to games.
In round 6 "book" said Lc0 to play 1. e4 rather than usual c4.
and in round 5 it said it to respond 1. .. c5 to 1. e4 rather than 1. .. e6.

In round 7 opponent played 1. d4, and we were immediately out of the "book". (it really contained nothing except 1. e4 c5)
Uri Blass
Posts: 10210
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: how will Leela fare at the WCCC?

Post by Uri Blass »

crem wrote: Sat Jul 21, 2018 5:43 pm
jp wrote: Fri Jul 20, 2018 6:47 am
yanquis1972 wrote: Thu Jul 19, 2018 11:50 pm if there was no book, it seems to me leela did awfully well in the opening.
What do you think of as "awfully well"? The Leela guys seem to be making the opposite claim. e.g. "Leela was handicapped by being -3 out of every opening" (not a real quote).
Who said that? That's not really true, Leela does well in opening, and in all WCCC games she went out of openings with both good self-eval and eval from other engines.

Out of entire Lc0 team, I think that an author of the blog is the only person who think that Lc0 would benefit from opening book, so there may be hints to that in the blog.
It is clear that LC0 can benefit from opening book.
A book always can be used to save time on the clock even if the engine can find the same moves without opening book.