LCZero: Progress and Scaling. Relation to CCRL Elo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Dann Corbit
Posts: 9901
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Dann Corbit » Fri Jun 01, 2018 5:32 am

Laskos wrote:
Wed Mar 28, 2018 2:19 pm
duncan wrote: I meant software which uses neural networks.is it giraffe ?
Giraffe, but it had a very different kind of learning and search.
I think if Giraffe were converted to Cuda, we might all get our eyes opened.
Oh, wait. They already did that.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

peter
Posts: 1762
Joined: Sat Feb 16, 2008 6:38 am
Full name: Peter Martan

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by peter » Fri Jun 01, 2018 6:36 am

Dann Corbit wrote:
Fri Jun 01, 2018 5:32 am
Laskos wrote:
Wed Mar 28, 2018 2:19 pm
duncan wrote: I meant software which uses neural networks.is it giraffe ?
Giraffe, but it had a very different kind of learning and search.
I think if Giraffe were converted to Cuda, we might all get our eyes opened.
Oh, wait. They already did that.
8-)
Peter.

Nay Lin Tun
Posts: 511
Joined: Mon Jan 16, 2012 5:34 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Nay Lin Tun » Fri Jun 01, 2018 7:11 am

If Leela were running in A0 hardware, Leela would be beating stockfish 8 on 32 cores. (64 thread) :mrgreen: :lol: :roll:

User avatar
George Tsavdaris
Posts: 1559
Joined: Thu Mar 09, 2006 11:35 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by George Tsavdaris » Fri Jun 01, 2018 7:55 am

I noticed yesterday that surprisingly Leela "solves" a good position.

White to play 1.Nd5!! and win. 1.Re4 just draws....
[d]8/2p2p1p/3k1p2/pn6/PNp4p/r3Rp1P/2P2P2/BK6 w - - 0 1

It finds Nd5!! :shock: :shock: A superb move! In a superb position(mate in 178).
But don't get too excited, as of course she finds it for wrong reasons and doesn't see it is winning. But she finds it and for someone that tries to solve a position the fact that Nd5 has so many visits and it's the preferable move of Leela, should ring a bell when analyzing a position that this move has its merits.
To be fair to other engines, Leela underestimates Re4 and thinks black is better, which is not the case as white puts much pressure to black with Re4, but can't win.



Good old [d] doesn't wok as it seems. :(
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

Milos
Posts: 3384
Joined: Wed Nov 25, 2009 12:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Milos » Fri Jun 01, 2018 11:20 am

Nay Lin Tun wrote:
Fri Jun 01, 2018 7:11 am
If Leela were running in A0 hardware, Leela would be beating stockfish 8 on 32 cores. (64 thread) :mrgreen: :lol: :roll:
Ofc it would, if those 32 cores were ARM cores running at 800MHz :roll: :roll:

JJJ
Posts: 1285
Joined: Sat Apr 19, 2014 11:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by JJJ » Fri Jun 01, 2018 1:09 pm

yanquis1972 wrote:
Thu May 31, 2018 11:02 pm
been watching a 1'1 match between leela & SF8 on one core, & it's transparently clear that, while tactics are an obvious issue, leela's woeful misevaluation of endgames is bleeding a lot of wins (& draws). i don't really understand how, after 15 million games, it could be so bad, but she has repeatedly dissolved into drawn endgames she thinks are +2 or +3. KRB vs KR is the last example off the top of my ahead, & i think the game eval was something like +2.50.

is there any concrete evidence this is improving?
Yeah, I see the exact same thing with Leela. But I guess with more training she will overcome that as well anyway.

Albert Silver
Posts: 2829
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Fri Jun 01, 2018 3:01 pm

yanquis1972 wrote:
Thu May 31, 2018 11:02 pm
been watching a 1'1 match between leela & SF8 on one core, & it's transparently clear that, while tactics are an obvious issue, leela's woeful misevaluation of endgames is bleeding a lot of wins (& draws). i don't really understand how, after 15 million games, it could be so bad, but she has repeatedly dissolved into drawn endgames she thinks are +2 or +3. KRB vs KR is the last example off the top of my ahead, & i think the game eval was something like +2.50.

is there any concrete evidence this is improving?
Well, for this to be a provable issue (compared to other top engines) you would need to show that other engines did better in the exact same time. Share a position where she made a mistake, entering the wrong endgame, and that SF or Komodo find the solution in that same time control.

You are aware that KRB vs KR is quite winnable, and depends very much on the defender having the right position and not making any mistakes even then? There are excellent practical chances to win it.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Milos
Posts: 3384
Joined: Wed Nov 25, 2009 12:47 am

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Milos » Fri Jun 01, 2018 3:15 pm

Albert Silver wrote:
Fri Jun 01, 2018 3:01 pm
yanquis1972 wrote:
Thu May 31, 2018 11:02 pm
been watching a 1'1 match between leela & SF8 on one core, & it's transparently clear that, while tactics are an obvious issue, leela's woeful misevaluation of endgames is bleeding a lot of wins (& draws). i don't really understand how, after 15 million games, it could be so bad, but she has repeatedly dissolved into drawn endgames she thinks are +2 or +3. KRB vs KR is the last example off the top of my ahead, & i think the game eval was something like +2.50.

is there any concrete evidence this is improving?
Well, for this to be a provable issue (compared to other top engines) you would need to show that other engines did better in the exact same time. Share a position where she made a mistake, entering the wrong endgame, and that SF or Komodo find the solution in that same time control.

You are aware that KRB vs KR is quite winnable, and depends very much on the defender having the right position and not making any mistakes even then? There are excellent practical chances to win it.
Lol, here is a test for you. Play against your strongest LC0 10s per move or even more if you wish. You with naked king and give LC0 bishop and knight. You can try couple of times. And please tell us the result.
Than take any below 2500 Elo engine without TBs and repeat. And tell us the result. ;)

Albert Silver
Posts: 2829
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Fri Jun 01, 2018 3:41 pm

Milos wrote:
Fri Jun 01, 2018 3:15 pm
Albert Silver wrote:
Fri Jun 01, 2018 3:01 pm
yanquis1972 wrote:
Thu May 31, 2018 11:02 pm
been watching a 1'1 match between leela & SF8 on one core, & it's transparently clear that, while tactics are an obvious issue, leela's woeful misevaluation of endgames is bleeding a lot of wins (& draws). i don't really understand how, after 15 million games, it could be so bad, but she has repeatedly dissolved into drawn endgames she thinks are +2 or +3. KRB vs KR is the last example off the top of my ahead, & i think the game eval was something like +2.50.

is there any concrete evidence this is improving?
Well, for this to be a provable issue (compared to other top engines) you would need to show that other engines did better in the exact same time. Share a position where she made a mistake, entering the wrong endgame, and that SF or Komodo find the solution in that same time control.

You are aware that KRB vs KR is quite winnable, and depends very much on the defender having the right position and not making any mistakes even then? There are excellent practical chances to win it.
Lol, here is a test for you. Play against your strongest LC0 10s per move or even more if you wish. You with naked king and give LC0 bishop and knight. You can try couple of times. And please tell us the result.
Than take any below 2500 Elo engine without TBs and repeat. And tell us the result. ;)
No need. It used to have trouble until it learned how to win them. All this has been tracked in detail:

https://docs.google.com/spreadsheets/d/ ... edit#gid=0
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

duncan
Posts: 10049
Joined: Mon Jul 07, 2008 8:50 pm

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by duncan » Fri Jun 01, 2018 4:14 pm

Albert Silver wrote:
Fri Jun 01, 2018 3:41 pm
Milos wrote:
Fri Jun 01, 2018 3:15 pm
Albert Silver wrote:
Fri Jun 01, 2018 3:01 pm


Well, for this to be a provable issue (compared to other top engines) you would need to show that other engines did better in the exact same time. Share a position where she made a mistake, entering the wrong endgame, and that SF or Komodo find the solution in that same time control.

You are aware that KRB vs KR is quite winnable, and depends very much on the defender having the right position and not making any mistakes even then? There are excellent practical chances to win it.
Lol, here is a test for you. Play against your strongest LC0 10s per move or even more if you wish. You with naked king and give LC0 bishop and knight. You can try couple of times. And please tell us the result.
Than take any below 2500 Elo engine without TBs and repeat. And tell us the result. ;)
No need. It used to have trouble until it learned how to win them. All this has been tracked in detail:

https://docs.google.com/spreadsheets/d/ ... edit#gid=0
do you know who can do it in least amount of moves stockfish or lc0?

Post Reply