Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Kanizsa
Posts: 53
Joined: Mon Feb 20, 2017 8:29 am
Location: Rialto, Venice

Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Kanizsa »

The TCEC world championship is always interesting. However, there's one decision that has left many chess fans like me scratching their heads: the choice of different networks for Leela and Kibitzer Leela.

In the superfinal, Leela is playing with an older network (number 6147500 I read) while Kibitzer Leela is using the most recent and strongest one, number 6315000. This seems like an odd decision, given that the most recent network is generally considered to be the stronger engine. According to Pohl's rating, modern 6315000 outperforms older 617500 by 11 Elo points, which is a significant margin nowadays.

To me, there are a few possible explanations for this choice.

One possibility is that the TCEC organizers are simply trying to experiment with different network configurations. They may be interested in seeing how Leela performs with an older network, and how Kibitzer Leela performs with the latest and strongest.

We don't know how this decision of different networks will impact the outcome of the superfinal. By giving to Leela a weaker network, they may be hoping to level the playing field and make the competition more exciting.

Whatever the reason, the decision to give Leela an outdated and weaker network is sure to be a topic of discussion among chess fans. Some will argue that it's an unfair advantage for Stockfish, while others will say that it's simply a way to make the competition more interesting.
AndrewGrant
Posts: 1811
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by AndrewGrant »

TCEC is not the one who decides which network plays. Leela team members would have made the submission you see there now. So your question is to them, not to TCEC. Perhaps the tests that say it was +11 did not exist at the time of submission. Perhaps those +11 results are not expected to be reflected under TCEC conditions. Perhaps the network did not even exist (!) at the time.

I took the post seriously, but I'll note there is a high chance of trolling, with statements like "they may be hoping to level the playing field and make the competition more exciting", when Stockfish has clearly dominated all recent TCEC events, despite the heavily dated hardware.
Friendly reminder that stealing is a crime, is wrong, and makes you a thief.
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Viz
Posts: 100
Joined: Tue Apr 09, 2024 6:24 am
Full name: Michael Chaly

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Viz »

Usual error bars for leela 11 elo gains is some 35 elo from my experience, so this also should be taken with a grain of salt.
Kanizsa
Posts: 53
Joined: Mon Feb 20, 2017 8:29 am
Location: Rialto, Venice

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Kanizsa »

AndrewGrant wrote: Thu May 16, 2024 2:22 pm TCEC is not the one who decides which network plays. Leela team members would have made the submission you see there now. So your question is to them, not to TCEC. Perhaps the tests that say it was +11 did not exist at the time of submission. Perhaps those +11 results are not expected to be reflected under TCEC conditions. Perhaps the network did not even exist (!) at the time.

I took the post seriously, but I'll note there is a high chance of trolling, with statements like "they may be hoping to level the playing field and make the competition more exciting", when Stockfish has clearly dominated all recent TCEC events, despite the heavily dated hardware.
Ok, thank you. I look forward to hearing someone, from the Leela team, to explain why Leela Blue's current network was not used in the final. Or, if you prefer, why Leela Blue as a Kibitzer don't use 6147500 network. Very, very strange decision.
Jouni
Posts: 3324
Joined: Wed Mar 08, 2006 8:15 pm

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Jouni »

Who has disadvantage? SF speed around 15 Mnps with 101 threads :o .
Jouni
AndrewGrant
Posts: 1811
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by AndrewGrant »

Jouni wrote: Thu May 16, 2024 6:34 pm Who has disadvantage? SF speed around 15 Mnps with 101 threads :o .
Well SF gets between 70mnps and 100mnps at CCC....
I assume your number is wrong... 15MNPS for 101 threads would be 150k nps per thread. That implies hardware problems.
Friendly reminder that stealing is a crime, is wrong, and makes you a thief.
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Jouni
Posts: 3324
Joined: Wed Mar 08, 2006 8:15 pm

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Jouni »

Number is correct. In current game between 10 and 20 Mnps.
Jouni
User avatar
Guenther
Posts: 4638
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Guenther »

AndrewGrant wrote: Thu May 16, 2024 6:48 pm
Jouni wrote: Thu May 16, 2024 6:34 pm Who has disadvantage? SF speed around 15 Mnps with 101 threads :o .
Well SF gets between 70mnps and 100mnps at CCC....
I assume your number is wrong... 15MNPS for 101 threads would be 150k nps per thread. That implies hardware problems.
Surprisingly the referee program Crystal (SF derivate) shows 80-90 Mnps at the same time...
https://rwbc-chess.de

trollwatch:
Talkchess nowadays is a joke - it is full of trolls/idiots/people stuck in the pleistocene > 80% of the posts fall into this category...
User avatar
pohl4711
Posts: 2486
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by pohl4711 »

AndrewGrant wrote: Thu May 16, 2024 6:48 pm
Jouni wrote: Thu May 16, 2024 6:34 pm Who has disadvantage? SF speed around 15 Mnps with 101 threads :o .
Well SF gets between 70mnps and 100mnps at CCC....
I assume your number is wrong... 15MNPS for 101 threads would be 150k nps per thread. That implies hardware problems.
The numbers are correct. I find the nps of Stockfish way to small in TCEC this year. Of course, the new nets made SF slower, but the TCEC-PC is (around) 50% of the ccc-hardware, so I would expect 40 mnps, not around 15-20, as we see right now in the TCEC superfinal. There is definitly something strange going on there.

Dont get me wrong: Stockfish running double fast would not have a measureable impact on Elo, if we consider the extremly long timecontrols (120min+12sec (!!!)). So, I am relativly convinced, Stockfish will win the superfinal anyway, even running way too slow.
Werewolf
Posts: 1833
Joined: Thu Sep 18, 2008 10:24 pm

Re: Why was Leela heavily disadvantaged in the TCEC championship with an outdated network?

Post by Werewolf »

I don't think there's a hardware fault. It's just the hardware isn't that fast:

"2x Xeon 6230R (52 cores/104 threads) 2.1 GHz"

This is Cascade Lake, which is basically Skylake, which shows virtually no advantage over Haswell.

A modern single-CPU Threadripper would wipe the floor with this, let alone dual EPYCs.