LCZero: Progress and Scaling. Relation to CCRL Elo

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
mhull
Posts: 12005
Joined: Wed Mar 08, 2006 8:02 pm
Location: Dallas, Texas
Full name: Matthew Hull

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by mhull » Thu May 24, 2018 3:13 pm

Albert Silver wrote:
Thu May 24, 2018 2:11 pm
... An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.
But what does hold water is this, that decisions based primarily on ever-increasing, self-discovered positional principles will eventually triumph altogether, which is the exact impression we have from Alpha Zero games -- the development of (for all intents and purposes) a God-like understanding of chess. This has been the promise of computer chess from the very beginning.
Matthew Hull

Kanizsa
Posts: 32
Joined: Mon Feb 20, 2017 7:29 am
Location: Rialto, Venice

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Kanizsa » Thu May 24, 2018 4:00 pm

Albert Silver wrote:
Thu May 24, 2018 2:11 pm
Kanizsa wrote:
Thu May 24, 2018 11:25 am

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.
Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?

Albert Silver
Posts: 2821
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Thu May 24, 2018 5:25 pm

Kanizsa wrote:
Thu May 24, 2018 4:00 pm
Albert Silver wrote:
Thu May 24, 2018 2:11 pm
Kanizsa wrote:
Thu May 24, 2018 11:25 am

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.
Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?
Capablanca was also joking. It is a tiresome question from know-nothing reporters that every World Champion has had to answer time and time again. Capablanca was a fantastic calculator.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

User avatar
Laskos
Posts: 9034
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Thu May 24, 2018 6:51 pm

Kanizsa wrote:
Thu May 24, 2018 11:25 am
Laskos wrote:
Sun Apr 01, 2018 9:37 am
Hi Robin!


LC0 seems already close to very strong engines in this opening suite. At this pace of advancement in positional understanding, I will be very curious how it develops.
Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.

Albert Silver
Posts: 2821
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Albert Silver » Thu May 24, 2018 7:16 pm

Laskos wrote:
Thu May 24, 2018 6:51 pm
Kanizsa wrote:
Thu May 24, 2018 11:25 am
Laskos wrote:
Sun Apr 01, 2018 9:37 am
Hi Robin!


LC0 seems already close to very strong engines in this opening suite. At this pace of advancement in positional understanding, I will be very curious how it develops.
Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.
Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

User avatar
Laskos
Posts: 9034
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Thu May 24, 2018 7:56 pm

Albert Silver wrote:
Thu May 24, 2018 7:16 pm
Laskos wrote:
Thu May 24, 2018 6:51 pm
Kanizsa wrote:
Thu May 24, 2018 11:25 am


Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.
Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view
Thanks, maybe you will keep me updated on these nets. I will test soon, one game here at LTC has to proceed to the end.

User avatar
Laskos
Posts: 9034
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Laskos » Thu May 24, 2018 10:32 pm

Laskos wrote:
Thu May 24, 2018 7:56 pm
Albert Silver wrote:
Thu May 24, 2018 7:16 pm
Laskos wrote:
Thu May 24, 2018 6:51 pm

No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.
Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view
Thanks, maybe you will keep me updated on these nets. I will test soon, one game here at LTC has to proceed to the end.
No, it performs worse by 3-4 positions on both tactical and positional 200 positions test suites. The speed is 2-2.2 times lower in my setup. Probably by now a bit lower result with larger net is expected, as the weights are from smaller nets. Hopefully it will improve soon.

Karlo Bala
Posts: 300
Joined: Wed Mar 22, 2006 9:17 am
Location: Novi Sad, Serbia

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Karlo Bala » Fri May 25, 2018 12:02 am

Kanizsa wrote:
Thu May 24, 2018 4:00 pm
Albert Silver wrote:
Thu May 24, 2018 2:11 pm
Kanizsa wrote:
Thu May 24, 2018 11:25 am

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?
Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.
Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?
No. It was not Capablanca who said that. It was Charles Jaffe.

‘Apropos is the story of the game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost. A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?” Here Jaffe explained: “I see only one move ahead, but always the best move.”

That is sufficient.’
Best Regards,
Karlo Balla Jr.

AlvaroBegue
Posts: 917
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by AlvaroBegue » Fri May 25, 2018 12:39 am

Karlo Bala wrote:
Fri May 25, 2018 12:02 am
Kanizsa wrote:
Thu May 24, 2018 4:00 pm
Albert Silver wrote:
Thu May 24, 2018 2:11 pm


Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.
Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?
No. It was not Capablanca who said that. It was Charles Jaffe.

‘Apropos is the story of the game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost. A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?” Here Jaffe explained: “I see only one move ahead, but always the best move.”

That is sufficient.’
That seems to not be right either. If you are interested in the origin of the quip, you should read this: http://www.chesshistory.com/winter/extr ... ahead.html

User avatar
Graham Banks
Posts: 32489
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: LCZero: Progress and Scaling. Relation to CCRL Elo

Post by Graham Banks » Sat May 26, 2018 10:08 pm

http://www.computerchess.org.uk/ccrl/40 ... 4-bit_w323

Leela Chess 0.10 64-bit w323 #110‑111 (2651 +19 −19)
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

Post Reply