LCZero: Progress and Scaling. Relation to CCRL Elo

[mhull]

... An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.

But what does hold water is this, that decisions based primarily on ever-increasing, self-discovered positional principles will eventually triumph altogether, which is the exact impression we have from Alpha Zero games -- the development of (for all intents and purposes) a God-like understanding of chess. This has been the promise of computer chess from the very beginning.

[Kanizsa]

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.

Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?

[Albert Silver]

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.

Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?

Capablanca was also joking. It is a tiresome question from know-nothing reporters that every World Champion has had to answer time and time again. Capablanca was a fantastic calculator.

[Laskos]

Hi Robin!


LC0 seems already close to very strong engines in this opening suite. At this pace of advancement in positional understanding, I will be very curious how it develops.

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.

[Albert Silver]

Hi Robin!


LC0 seems already close to very strong engines in this opening suite. At this pace of advancement in positional understanding, I will be very curious how it develops.

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.

Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view

[Laskos]

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.

Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view

Thanks, maybe you will keep me updated on these nets. I will test soon, one game here at LTC has to proceed to the end.

[Laskos]

No, but I am now using a different setup and hardware. cuDNN engine Lc0, and different TC. Also, both SF and Komodo improved substantially since the earlier versions, especially SF, but LCZero nets seem to have even regressed since NN240s on this positional test suite. I remember I got a maximum of 128/200 solved, now only 123/200. Here are the results for top engines with 10s/position (standard engines on 4 i7 cores, Lc0 cuDNN on GTX 1060 GPU):

Openings200 positional test suite:

SF dev:
score=139/200 [averages on correct positions: depth=14.0 time=1.23 nodes=6174345]

Komodo 12:
score=138/200 [averages on correct positions: depth=13.6 time=0.94 nodes=4600354]

Lc0:
score=123/200 [averages on correct positions: depth=1.1 time=1.78 nodes=5771]

Other strong engines are below Lc0, for example Houdini 1.5a on 4 threads is significantly below:

Houdini 1.5a:
score=98/200 [averages on correct positions: depth=10.3 time=0.88 nodes=6513332]

All in all, Lc0 is still very strong on this suite, but SF and Komodo are to reckon with. You have to keep in mind that the opening suite is not purely positional, some sort of search is always involved, I just tried to minimize its impact by using databases of human openings and outcomes, checking at the same time with engines for no tactical fluctuations.

Have you tried one of the test 20x256 nets? Here is the latest one posted today:

https://drive.google.com/file/d/11ueg-m ... fPjw6/view

Thanks, maybe you will keep me updated on these nets. I will test soon, one game here at LTC has to proceed to the end.

No, it performs worse by 3-4 positions on both tactical and positional 200 positions test suites. The speed is 2-2.2 times lower in my setup. Probably by now a bit lower result with larger net is expected, as the weights are from smaller nets. Hopefully it will improve soon.

[Karlo Bala]

Hi Kai,
what's about your last experiments with this opening suite ?
Are last nets of LC0 (those >300) positionally better than Stockfish & Komodo ?

Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.

Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?

No. It was not Capablanca who said that. It was Charles Jaffe.

‘Apropos is the story of the game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost. A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?” Here Jaffe explained: “I see only one move ahead, but always the best move.”

That is sufficient.’

[AlvaroBegue]

Offhand, I'd say maybe, but that is a very speculative maybe. One cannot remove tactics from the equation, so oversights in its calculations will affect its decisions. An argument such as ''this would be a great move if.... it didn't lose a piece" holds no water in my book.

Once Capablanca was asked: how many moves ahead can you see ?
Capablanca said he can see only 1 move ahead, but confidently he was sure this was the best one.

I'm asking myself: Is now LCZ 325 the Great chess-software of all time at depth 0, like Capablanca was for sure ?

No. It was not Capablanca who said that. It was Charles Jaffe.

‘Apropos is the story of the game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost. A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?” Here Jaffe explained: “I see only one move ahead, but always the best move.”

That is sufficient.’

That seems to not be right either. If you are interested in the origin of the quip, you should read this: http://www.chesshistory.com/winter/extr ... ahead.html

[Graham Banks]

http://www.computerchess.org.uk/ccrl/40 ... 4-bit_w323

Leela Chess 0.10 64-bit w323 #110‑111 (2651 +19 −19)