Strange Lc0 TCEC performance

[Werewolf]

Nvidia 11 series is just 2-3 months from now, and as we know, they will come with some massive improvements.

The stats I've seen suggest "only" +15%

But I was amazed Lc0 drew with Houdini 6 on 8 cores in a mini-match I ran yesterday on my 1060 card. It's an interesting time for computer chess...

[kranium]

think it's in the rules that tweaks can be made within season if a play-affecting issue is found.

No. Tweaks to the engines are not allowed.
Tweaks to the hardware for engines are. E.g lowering cores.

Here's the rule:

"Critical Engine Bugs
In the case of a serious, play-limiting bug (like crashing or interface communication problems, not including losses on time) not discovered during the pre-Season testing, the engine can be updated once per Division to fix this/these bug/bugs only. If this update still doesn't fix the problem(s) or if there is no update available, the engine might have the number of cores reduced, have the hash size reduced or have the tablebase access disabled - these changes will remain for the rest of the Division."

The key words are: "play-limiting" (crashing or interface communication problems)
Clearly this was not the case for Lc0 and DeusX...

PS George-
lowering cores means changing the engine config, reducing ('tweaking') the UCI parameter for that particular engine, not for the entire system

[Milos]

Again spitting gibberish with some 5% here and there, while missing the general picture? Nvidia 11 series is just 2-3 months from now, and as we know, they will come with some massive improvements. And by the way, my OC i7 4790 is at least on par with say stock i7 7700. All in all, you nitpick (with misplaced wording) on the general picture that on fairly balanced CPU/GPU home hardware, Lc0 with a good net is on par with SF8 on 1-2 threads in blitz. And probably soon will be on par with SF8 on full CPU, and one could casually say "Lc0 is SF8 level". Generally, behave, and don't use misplaced language.

Sorry but this childish fanboy behaviour of yours, patronising and telling ppl what to do is not gonna work.
You simply ignore or disregard facts and offer no arguments.
The only true claim of yours is that Lc0 on 1060 is on par or maybe slightly better (depending on actual TC) than SF8 on a single strong core in blitz. Everything else is, either gross exaggeration or just invented stuff.
You talk about massive GPU improvements that are gonna be at best 15% and ignore the fact that moneywise your 1060 is on par only to 8 core/16 threads Ryzen that gives 2.5x more nps than your 5 years old CPU, and powerwise 1060 is on the level of 16 core/32 threads Threadripper.
To me it seems you are just acting angry coz your precious CPU (with only ancient 4 cores) is much, much worse than some much cheaper AMDs.

And you talk about some improvement in Lc0 that would magically enable it to go from on par on single core to on par with 4 cores while (and I tested that thoroughly) the glorified 10780 net has 5% worse performance than 10445 vs SFdev in blitz, i.e. there is clear regression and 20x256 nets have saturated. Again clear case of fanboyism on your side.
Are you even aware that strongest Lc0 on 1060 can't beat Droidfish on 3 years old Samsung Galaxy S6?
Hey, Lc0 on state-of-the-art middle range GPU is not better than old SF (SF9) on old cellphone!!! So much about equal hardware and other bullshit.

[CMCanavessi]

think it's in the rules that tweaks can be made within season if a play-affecting issue is found.

No. Tweaks to the engines are not allowed.
Tweaks to the hardware for engines are. E.g lowering cores.

Here's the rule:

"Critical Engine Bugs
In the case of a serious, play-limiting bug (like crashing or interface communication problems, not including losses on time) not discovered during the pre-Season testing, the engine can be updated once per Division to fix this/these bug/bugs only. If this update still doesn't fix the problem(s) or if there is no update available, the engine might have the number of cores reduced, have the hash size reduced or have the tablebase access disabled - these changes will remain for the rest of the Division."

The key words are: "play-limiting" (crashing or interface communication problems)
Clearly this was not the case for Lc0 and DeusX...

PS George-
lowering cores means changing the engine config, reducing ('tweaking') the UCI parameter for that particular engine, not for the entire system

That rule talks about "engine bugs", which clearly wasn't the case here. It was a server issue, not an engine(s) issue. I think it's sad that Arasan will promote only because it was given a free point when Nemorino was about to beat it and crashed + was helped by the server issue that affected both Leela and lc0. The whole purpose of the tournament is that stronger engines should play the highest leagues. Here, clearly, lc0 and Ethereal were the strongest engines, by far, yet, because the server was misconfigured/poorly cooled/whatever, an engine will not advance. There wasn't anything the devs could do.

[Milos]

That rule talks about "engine bugs", which clearly wasn't the case here. It was a server issue, not an engine(s) issue. I think it's sad that Arasan will promote only because it was given a free point when Nemorino was about to beat it and crashed + was helped by the server issue that affected both Leela and lc0. The whole purpose of the tournament is that stronger engines should play the highest leagues. Here, clearly, lc0 and Ethereal were the strongest engines, by far, yet, because the server was misconfigured/poorly cooled/whatever, an engine will not advance. There wasn't anything the devs could do.

Oh cry me a river. So much whining. Like kids. Ppl really get this too personally.
You didn't even say a word when Ivanhoe was playing with a wrong binary (sent by competitor with agenda against Ivanhoe) that was never ever tested by TCEC, even though TCEC in the past already had a properly working binry. With working Ivanhoe Lc0 would anyway never promote to Div2 since it is most certainly weaker than Ivanhoe on 43 cores in TCEC conditions (that is of a strength of Gull).
So go cry somewhere else please. This is just ridiculous.

[CMCanavessi]

That rule talks about "engine bugs", which clearly wasn't the case here. It was a server issue, not an engine(s) issue. I think it's sad that Arasan will promote only because it was given a free point when Nemorino was about to beat it and crashed + was helped by the server issue that affected both Leela and lc0. The whole purpose of the tournament is that stronger engines should play the highest leagues. Here, clearly, lc0 and Ethereal were the strongest engines, by far, yet, because the server was misconfigured/poorly cooled/whatever, an engine will not advance. There wasn't anything the devs could do.

Oh cry me a river. So much whining. Like kids. Ppl really get this too personally.
You didn't even say a word when Ivanhoe was playing with a wrong binary (sent by competitor with agenda against Ivanhoe) that was never ever tested by TCEC, even though TCEC in the past already had a properly working binry. With working Ivanhoe Lc0 would anyway never promote to Div2 since it is most certainly weaker than Ivanhoe on 43 cores in TCEC conditions (that is of a strength of Gull).
So go cry somewhere else please. This is just ridiculous.

Where did I say that lc0 should advance instead of Arasan? Learn to read.
And also, even if you put ASMFish in div4, leela still promotes, because top 2 do so. Learn the rules.

[Hai]

N Engine Rtng Pts Gm SB X Elo Perf Et Lc Ar Pe De Ne Ha Bo

1 Ethereal 10.81 3176 22.5 28 281.50 0 +236 80.4 ···· 1=== 1=11 ===1 1==1 1111 11=1 111=
2 lc0 16.10520 3219 16.0 28 204.00 0 -33 57.1 0=== ···· ==0= ==1= ==1= =011 ===1 1==1

3 Arasan TCEC13 3142 16.0 28 196.00 0 +77 57.1 0=00 ==1= ···· ==10 01=1 1=1= ===1 1==1
4 Pedone 1.8 3104 15.0 28 186.75 0 +100 53.6 ===0 ==0= ==01 ···· 1000 0=== 1111 =111
5 DeusX 1.0 3200 13.5 28 169.00 0 -86 48.2 0==0 ==0= 10=0 0111 ···· =10= ==00 1=11
6 Nemorino 5.01 3104 12.0 28 143.50 2 +4 42.9 0000 =100 0=0= 1=== =01= ···· 1=0= 0111
7 Hannibal 20180806 3193 12.0 28 141.25 0 -124 42.9 00=0 ===0 ===0 0000 ==11 0=1= ···· 11=1
8 Bobcat 8 3072 5.0 28 75.50 0 -175 17.9 000= 0==0 0==0 =000 0=00 1000 00=0 ····

Due to the fact that Leela was crippled, how long does it take to see the 4 games vs Arasan to be replayed?
To decide which one will go to division 2?

Without crippled hardware, Leela would be on first place.
Deux maybe second place!

99% will go offline without NN engines at TCEC.

[sovaz1997]

Milos, restarting games in TCEC would show you the real performance of LCZero. You do not seem to understand.

IvanHoe with these settings sent the author. Although I would have set up the work on 43 cores at the place of the organizers.

[Milos]

That rule talks about "engine bugs", which clearly wasn't the case here. It was a server issue, not an engine(s) issue. I think it's sad that Arasan will promote only because it was given a free point when Nemorino was about to beat it and crashed + was helped by the server issue that affected both Leela and lc0. The whole purpose of the tournament is that stronger engines should play the highest leagues. Here, clearly, lc0 and Ethereal were the strongest engines, by far, yet, because the server was misconfigured/poorly cooled/whatever, an engine will not advance. There wasn't anything the devs could do.

Oh cry me a river. So much whining. Like kids. Ppl really get this too personally.
You didn't even say a word when Ivanhoe was playing with a wrong binary (sent by competitor with agenda against Ivanhoe) that was never ever tested by TCEC, even though TCEC in the past already had a properly working binry. With working Ivanhoe Lc0 would anyway never promote to Div2 since it is most certainly weaker than Ivanhoe on 43 cores in TCEC conditions (that is of a strength of Gull).
So go cry somewhere else please. This is just ridiculous.

Where did I say that lc0 should advance instead of Arasan? Learn to read.
And also, even if you put ASMFish in div4, leela still promotes, because top 2 do so. Learn the rules.

You said "it is sad that Arasan will promote" as something unfair and you also said that "lc0 and Ethereal were the strongest engines, by far", both of the claims frankly total bullshit (Ethereal is clearly the strongest but Lc0 certainly not). And then you are asking where did you say Lc0 should advance instead of Arasan??? I know English is not your first language, but seems you are really lacking some basic comprehension at least functional one.
You seems so angry that you can't even read properly. I clearly wrote that Lc0 wouldn't promote to Div2 because Ethereal and Ivanhoe would promote instead.
You should try to calm down, or if you can't go somewhere cry a bit and then after you feel a bit less sad and angry come back to comment .

[sovaz1997]

That rule talks about "engine bugs", which clearly wasn't the case here. It was a server issue, not an engine(s) issue. I think it's sad that Arasan will promote only because it was given a free point when Nemorino was about to beat it and crashed + was helped by the server issue that affected both Leela and lc0. The whole purpose of the tournament is that stronger engines should play the highest leagues. Here, clearly, lc0 and Ethereal were the strongest engines, by far, yet, because the server was misconfigured/poorly cooled/whatever, an engine will not advance. There wasn't anything the devs could do.

Oh cry me a river. So much whining. Like kids. Ppl really get this too personally.
You didn't even say a word when Ivanhoe was playing with a wrong binary (sent by competitor with agenda against Ivanhoe) that was never ever tested by TCEC, even though TCEC in the past already had a properly working binry. With working Ivanhoe Lc0 would anyway never promote to Div2 since it is most certainly weaker than Ivanhoe on 43 cores in TCEC conditions (that is of a strength of Gull).
So go cry somewhere else please. This is just ridiculous.

Where did I say that lc0 should advance instead of Arasan? Learn to read.
And also, even if you put ASMFish in div4, leela still promotes, because top 2 do so. Learn the rules.

You said "it is sad that Arasan will promote" as something unfair and you also said that "lc0 and Ethereal were the strongest engines, by far", both of the claims frankly total bullshit (Ethereal is clearly the strongest but Lc0 certainly not). And then you are asking where did you say Lc0 should advance instead of Arasan??? I know English is not your first language, but seems you are really lacking some basic comprehension at least functional one.
You seems so angry that you can't even read properly. I clearly wrote that Lc0 wouldn't promote to Div2 because Ethereal and Ivanhoe would promote instead.
You should try to calm down, or if you can't go somewhere cry a bit and then after you feel a bit less sad and angry come back to comment .

Lol you really think that IvanHoe on 43 cores stronger than Lc0?