Where are the funs of Leela?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Alexander Schmidt
Posts: 1086
Joined: Thu May 10, 2007 12:49 pm

Re: Where are the funs of Leela?

Post by Alexander Schmidt » Sat Jun 08, 2019 5:19 am

corres wrote:
Thu Jun 06, 2019 9:10 pm
As I see basically Leela is weaker against the relative weaker engines than the Stockfish family are.
Another Leela's issue is as the opening position is more further from start position as Leela is more and more weaker. It is a consequence of self-play training what begins from start position.
You got two faces. That's annoying. :roll:

corres
Posts: 1398
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Where are the funs of Leela?

Post by corres » Sat Jun 08, 2019 8:55 am

Alexander Schmidt wrote:
Sat Jun 08, 2019 5:19 am
corres wrote:
Thu Jun 06, 2019 9:10 pm
As I see basically Leela is weaker against the relative weaker engines than the Stockfish family are.
Another Leela's issue is as the opening position is more further from start position as Leela is more and more weaker. It is a consequence of self-play training what begins from start position.
You got two faces. That's annoying.
I am not a fan of Leela and a fan of Stockfish either.
But I use both of engines.
What is "annoying" for you?

daws
Posts: 909
Joined: Wed Mar 08, 2006 9:03 pm

Re: Where are the funs of Leela?

Post by daws » Sat Jun 08, 2019 11:56 pm

Alexander Schmidt wrote:
Thu Jun 06, 2019 4:58 pm

An 3000 ELO Engine will performe better against a older version of itself with 2990 ELO than against a different engine with 2990 ELO. Thus it is not a surprise that in this setup some Stockfish engines took the lead.
Not sure if this explanation quite works when four out of the six Stockfish engines are currently performing better than LC0.

Dann Corbit
Posts: 9829
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Where are the funs of Leela?

Post by Dann Corbit » Sun Jun 09, 2019 12:11 am

Try a mind experiment.
Take lc0 and sf and assume that on some hardware they are about the same strength.
Outcome is easy to imagine. It's fifty-fifty on who will win a multi game contest.
Now, try two sf verses lc0.
To win the contest, lc0 must beat sf0 AND sf1. The word AND here is a magic helper for us, because it means that lc0 must beat both opponents to win.
Lc0 no longer has a 50% chance of winning the contest.
And sf has a very good chance to win it now.
Now, let's enter 1000 sf engines.
At this point lc0 has very close to zero chance of winning.
It does not matter if they have exactly the same strength. Sf will almost surely win the contest.
That is a big reason why we see contests that do not allow multiple variants of the same engine.
Even if you do not understand the exact probabilities, it should be easy for anyone to understand that this is not fair.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Alexander Schmidt
Posts: 1086
Joined: Thu May 10, 2007 12:49 pm

Re: Where are the funs of Leela?

Post by Alexander Schmidt » Sun Jun 09, 2019 7:00 am

daws wrote:
Sat Jun 08, 2019 11:56 pm
Alexander Schmidt wrote:
Thu Jun 06, 2019 4:58 pm
An 3000 ELO Engine will performe better against a older version of itself with 2990 ELO than against a different engine with 2990 ELO. Thus it is not a surprise that in this setup some Stockfish engines took the lead.
Not sure if this explanation quite works when four out of the six Stockfish engines are currently performing better than LC0.
I guess the same applies to the A/B engines compared to one nn engine. All A/B engines are very similar to each other, just Lc0 is completely different. So it is likely that the good A/B engines perform better against weak A/B engines than Lc0. But there are really not enough games to draw any serious conclusion...

daws
Posts: 909
Joined: Wed Mar 08, 2006 9:03 pm

Re: Where are the funs of Leela?

Post by daws » Sun Jun 09, 2019 7:34 am

Dann Corbit wrote:
Sun Jun 09, 2019 12:11 am
Try a mind experiment.
Take lc0 and sf and assume that on some hardware they are about the same strength.
Outcome is easy to imagine. It's fifty-fifty on who will win a multi game contest.
Now, try two sf verses lc0.
To win the contest, lc0 must beat sf0 AND sf1. The word AND here is a magic helper for us, because it means that lc0 must beat both opponents to win.
Lc0 no longer has a 50% chance of winning the contest.
And sf has a very good chance to win it now.
Now, let's enter 1000 sf engines.
At this point lc0 has very close to zero chance of winning.
It does not matter if they have exactly the same strength. Sf will almost surely win the contest.
That is a big reason why we see contests that do not allow multiple variants of the same engine.
Even if you do not understand the exact probabilities, it should be easy for anyone to understand that this is not fair.
I would agree with your mind experiment if all the SF engines are the same strength. In the current TCEC tournament five of the six SF engines are older and relatively weaker versions and are therefore easier to score points against by either the latest SF or LC0. For this reason, I don't think your mind experiment can be applied to this particular tournament.

corres
Posts: 1398
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Where are the funs of Leela?

Post by corres » Sun Jun 09, 2019 7:42 am

Dann Corbit wrote:
Sun Jun 09, 2019 12:11 am
Try a mind experiment.
Take lc0 and sf and assume that on some hardware they are about the same strength.
Outcome is easy to imagine. It's fifty-fifty on who will win a multi game contest.
Now, try two sf verses lc0.
To win the contest, lc0 must beat sf0 AND sf1. The word AND here is a magic helper for us, because it means that lc0 must beat both opponents to win.
Lc0 no longer has a 50% chance of winning the contest.
And sf has a very good chance to win it now.
Now, let's enter 1000 sf engines.
At this point lc0 has very close to zero chance of winning.
It does not matter if they have exactly the same strength. Sf will almost surely win the contest.
That is a big reason why we see contests that do not allow multiple variants of the same engine.
Even if you do not understand the exact probabilities, it should be easy for anyone to understand that this is not fair.
I have only one (crossing) question:
What is the possibility of that FOUR Stockfish-s stand before Leela?

corres
Posts: 1398
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Where are the funs of Leela?

Post by corres » Sun Jun 09, 2019 7:48 am

Alexander Schmidt wrote:
Sun Jun 09, 2019 7:00 am
I guess the same applies to the A/B engines compared to one nn engine. All A/B engines are very similar to each other, just Lc0 is completely different. So it is likely that the good A/B engines perform better against weak A/B engines than Lc0. But there are really not enough games to draw any serious conclusion...
So you think Leela is an offering of Stockfis' "conspiracy"?
I think you would search for obvious issues of Leela instead of these artificial theories.

Alexander Schmidt
Posts: 1086
Joined: Thu May 10, 2007 12:49 pm

Re: Where are the funs of Leela?

Post by Alexander Schmidt » Mon Jun 10, 2019 8:11 pm

corres wrote:
Sun Jun 09, 2019 7:48 am
Alexander Schmidt wrote:
Sun Jun 09, 2019 7:00 am
I guess the same applies to the A/B engines compared to one nn engine. All A/B engines are very similar to each other, just Lc0 is completely different. So it is likely that the good A/B engines perform better against weak A/B engines than Lc0. But there are really not enough games to draw any serious conclusion...
So you think Leela is an offering of Stockfis' "conspiracy"?
I think you would search for obvious issues of Leela instead of these artificial theories.
Conspiracy? What are you talking about? You asked for opinions, and what I wrote is not just an "artificial theory". It's a logical conclusion of someone with 30 years of experiences in computer chess.

Your answer is a piece of impudence. That's what I mean with your 2 faces. First asking a question and then briefing against well-intentioned answers. Don't worry, I won't do this again.

corres
Posts: 1398
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Where are the funs of Leela?

Post by corres » Mon Jun 10, 2019 10:13 pm

Alexander Schmidt wrote:
Mon Jun 10, 2019 8:11 pm
corres wrote:
Sun Jun 09, 2019 7:48 am
Alexander Schmidt wrote:
Sun Jun 09, 2019 7:00 am
I guess the same applies to the A/B engines compared to one nn engine. All A/B engines are very similar to each other, just Lc0 is completely different. So it is likely that the good A/B engines perform better against weak A/B engines than Lc0. But there are really not enough games to draw any serious conclusion...
So you think Leela is an offering of Stockfis' "conspiracy"?
I think you would search for obvious issues of Leela instead of these artificial theories.
Conspiracy? What are you talking about? You asked for opinions, and what I wrote is not just an "artificial theory". It's a logical conclusion of someone with 30 years of experiences in computer chess.

Your answer is a piece of impudence. That's what I mean with your 2 faces. First asking a question and then briefing against well-intentioned answers. Don't worry, I won't do this again.
My answer is not impudence but challenging.
As you are so experienced I hope you will analyze the weakness of Leela.
Because these are the main causes of Leela's lag and not the superiority of Stockfish over the weaker AB engines. Moreover the superiority of Stockfish may arise from the dynamic contempt what lacks from Leela.
Btw.
Leela also lost two points against the four Stockfish what stand before Leela.

Post Reply