Strange Lc0 TCEC performance

[frankp]

This is slightly weird Chris.
As I said my interpretation of comments I have seen here and elsewhere.
You may disagree. Fine.
Lets just say I am mistaken.

[chrisw]

This is slightly weird Chris.
As I said my interpretation of comments I have seen here and elsewhere.
You may disagree. Fine.
Lets just say I am mistaken.

If you wrote "antipathy towards the hype/mania", I'ld be agreeing with you.

[Laskos]

AB engines are very successful mimics of the 'human' approach to chess, hard coded into their program. It would make sense for them to incorporate the AI approach if this becomes superior - positional evaluation etc.

I am still at the fascination stage with leela, where you can give a program the rules of chess, a NN structure and let it learn for itself from self-play to become a very strong chess entity in a relatively short time - without humans having to tell it what to do at every stage based on decades of research into search techniques and encoding positional evaluation.

(Hard to understand the antipathy towards the project, particularly on this forum.).

Please quote, from one smart person on this forum, the raw data that says there is "antipathy to the project". Else Strawman Fallacy.

This is irony - yes? Or do you expect me to define who and who is not smart on this forum?
Whichever, my interpretation on some comments seen here - right or wrong as it may be. (And of course comments elsewhere.). And btw, not you. YMMV.

By smart, I mean excluding anyone commenting "Leela yah boo sucks", for example. I don't think those exist here either.

There isn't antipathy towards the project. One quote would be enough. Do you have one to show us?

From memory:
Daniel claimed that Leela is 2200 CCRL Elo level, but that was on one core of CPU. At that time, on reasonable GPU it was already above 2800 CCRL in blitz conditions.
Andrew days ago wanted to see Leela on CPU, as GPU is a very specialized piece of hardware only freaks of "Call of Duty" use, not the many millions of chess engines enthusiasts.
Milos called me a "fanboy" of Leela, although I was one of the first to point out the limitations of Leela DCNN approach as one very much task specialized and not in any way approaching abstraction and generalization of a flexible AI.

Just from memory on my phone (not easy to search or type anything).

[AndrewGrant]

Watch a game of TCEC -- Leela best moves will match the opponents predictions for large chunks of the game.

This is clearly misleading since this happens with all pairs of strong opponents.
And yet after only 6 games in the bonus games between Leela 10520 and 10800 against Ethereal 10.81 with 6 draws out of 6 games where Leela in all controlled the game, in 2 it completely outplayed Ethereal in the middlegame.
So how to say Ethereal matched Leela's predictions when it got outplayed?

Leela 10520 - Ethereal 10.81



Ethereal 10.81 - Leela 10809


And this was full 43 cores TCEC Ethereal, while Leela was struggling playing with 1 GPU due to the thermal problems!
8000$ versus 900$ hardware in favor of Ethereal.

In the aforementioned 2 games i said, Leela didn't won since she couldn't find the winning moves in the endgame:
In the one game the 49.Rd7! was winning much easier instead of 49.R2d5, but still this was also winning and Leela missed 54.Bf5 that would win and played 54.Kd5 with the Kc6? plan that draws.
And in the other game 48....g5+ would win instead of Ra1?!


The games (they contain also the 3 Leela - SF8 43-cores bonus games that were also all draws)
https://drive.google.com/open?id=1AhNm0 ... CyW8BPa_iD

Big out play followed by a draw ... hmmm

Also, $8k is an awful talking point. For $800 I get 70% of TCEC NPS -- with the newer Ryzens you can get 100%, maybe more, with < $2k.

Oh ... and Leela needs a CPU too you know. Much more than a single $800 GPU...

[AndrewGrant]

Andrew days ago wanted to see Leela on CPU, as GPU is a very specialized piece of hardware only freaks of "Call of Duty" use, not the many millions of chess engines enthusiasts.

Yeah the first part of that statement is something I said.

I also never talked about "Call of Duty", so please ask the mods to correct your posting to reflect my actual words.

I don't think there are millions of chess engine enthusiasts, also.

[Laskos]

Andrew days ago wanted to see Leela on CPU, as GPU is a very specialized piece of hardware only freaks of "Call of Duty" use, not the many millions of chess engines enthusiasts.

Yeah the first part of that statement is something I said.

I also never talked about "Call of Duty", so please ask the mods to correct your posting to reflect my actual words.

I don't think there are millions of chess engine enthusiasts, also.

You don't get it, right? To re-phrase for clarity: there are hundreds of chess engine enthusiasts using chess engines on CPUs with more than 4 cores. There are many millions of gamers using GPUs equal or above Nvidia 960.

[AndrewGrant]

Andrew days ago wanted to see Leela on CPU, as GPU is a very specialized piece of hardware only freaks of "Call of Duty" use, not the many millions of chess engines enthusiasts.

Yeah the first part of that statement is something I said.

I also never talked about "Call of Duty", so please ask the mods to correct your posting to reflect my actual words.

I don't think there are millions of chess engine enthusiasts, also.

You don't get it, right? To re-phrase for clarity: there are hundreds of chess engine enthusiasts using chess engines on CPUs with more than 4 cores. There are many millions of gamers using GPUs equal or above Nvidia 960.

I understood your intent, but you suggested I said things I did not. Thats wrong Laskos.

[frankp]

I am still on the page that a self-trained neural network (probably not optimised for the task) given ‘only the rules of chess’ can in a few months hold its own against good AB engines, which are based on decades of research and development optimising search and the evaluation of terminal nodes.

The fact that such an entity cannot be run very well on a cpu but needs a graphics card instead seems frankly irrelevant, other than to an individual who may want to run it on their own pc at close to full strength.

This thread was started by Leo, who said:

“I have had major doubts about Lc0 from the beginning. It reminds me of quantum computing. It has all this potential but is still years away from emerging. " 1 bird in the hand is better than 2 in the bush's."

I tend to disagree. The hardware already exists to run Lc0 at a ‘very high’ level, will get better quickly, as, I expect, so will Lc0.

This is new territory, but it will be interesting to see if Lc0 can surpass A0's ‘flat-lining’ when it moves to bigger networks as the hardware becomes available to make this possible. Will pure AB engines be able to match this rate of development. I guess we will find out.

It will be interesting to look back on these ‘discussions’ in a few years time.

[jkiliani]

This is new territory, but it will be interesting to see if Lc0 can surpass A0's ‘flat-lining’ when it moves to bigger networks as the hardware becomes available to make this possible. Will pure AB engines be able to match this rate of development. I guess we will find out.

It will be interesting to look back on these ‘discussions’ in a few years time.

I still remember those threads from 6 months ago saying that Leela is playing "Orang-Utan chess" (which she was, at the time), and that she would never learn to play decently (which she undoubtedly does now)

Let's revisit this a year from now.

[chrisw]

This is new territory, but it will be interesting to see if Lc0 can surpass A0's ‘flat-lining’ when it moves to bigger networks as the hardware becomes available to make this possible. Will pure AB engines be able to match this rate of development. I guess we will find out.

It will be interesting to look back on these ‘discussions’ in a few years time.

I still remember those threads from 6 months ago saying that Leela is playing "Orang-Utan chess" (which she was, at the time), and that she would never learn to play decently (which she undoubtedly does now)

Let's revisit this a year from now.

Development progress or not is absolutely not any kind of function of negative criticism, so implications of progress by reference to probably very uneducated critics is no cigar.

On the other hand, positive hype and crowd based positive hype, unrealistic expectations, wild results reporting, won games video reporting, particularly in own group, create an atmosphere where developers social group exposed start making mistakes. So, critiqueing the negative critics is absolutely the wrong target. Shutting up the crowd gone mad would be much more effective.