Lc0 wins TCEC Cup 11 Final

[Werewolf]

GPU hardware is improving at a much faster rate than CPU it seems. So the future of Lc0 is looking good, apart from the huge power consumption of the high-end cards.

Use MacBook Pro M1 MAX 32 GPU cores. Or soon 64 GPU cores.

Using the recommended best big net, T78, my MacBook Air M2 gets a whopping 300 nps. My PC with my 3090 card gets 20000 nps. The 4090 would be around 35000 nps.

Apple just sucks for chess.

It was your mistake to buy an Air.
Which net size?

Are you talking about this: https://github.com/LeelaChessZero/lc0/issues/1562
LC0 is reaching 0.3K now.
5x faster = 1.5K with 8 GPU cores
But the Apple M1X/ will use 32 GPU cores.
1.5K x 4 = 6K !!

Have you seen the other results:
https://github.com/LeelaChessZero/lc0/issues/1795
Big LC0 806286 net
GPU + 2 CPU cores:
Total time (ms) : 350806
Nodes searched : 498436
Nodes/second : 1421
GPU + 1 CPU core:
Total time (ms) : 350752
Nodes searched : 498331
Nodes/second : 1421
That's without the 5x faster improvements.
There was a match some months ago where this net lost only 4 and won 1 game against newest Stockfish. The rest were a draw.
With the new LC0 nets, which are much better than T80 nets, I would expect a tie.
Yes you are right, Apple just sucks for chess

It reaches on my MacBook 1.5K.
If the LC0 developers would start to improve LC0 on Apple devices, then LC0 would reach 7.5K.
With the newest nets, LC0 should win a match against Stockfish.
Don't compare RTX watt consumption to Apple GPU if you have an Nvidia RTX, because it's safer for your life.

I was talking about this net:
https://github.com/LeelaChessZero/lc0/w ... ts-for-Lc0 at the top under large nets. It's a T78 and 512x20 in size.

The M1 Max has a 32 core GPU producing 10.6 TFLOPS. The M2 has a 10 core GPU producing 3.6 TFLOPS. So the M1 Max is 2.94 times faster.
If I get 300 nps you should get around 900.

That's less than 1/20th of the speed of a 3090 and around 1/40th the speed of a 4090.
The issues of catching fire etc seem to be mostly resolved now by Nvidia.

Maybe you value wattage above all things, personally I value time much more highly. I can get done in 3 seconds what takes the M1 Max a whole minute.

[dkappe]

Can we take this Apple bashing to its own thread? It has nothing to do with TCEC Cup 11.

[wickedpotus]

Can we take this Apple bashing to its own thread? It has nothing to do with TCEC Cup 11.

The thread went from LC0 doing well to Networks to network-architecture to training to GPU and then some "magnum" derailed it even more by promoting Apples not so great GPU as a "solution"and then many corrected this missinformation. I would hardly call this bashing... But I agree with your view that we should get back on track to discuss TCEC or close this thread.

[abgursu]

Whenever lc0 is mentioned in any kind of topic it eventually comes down to the hardware talk.

[Werewolf]

Just to be clear I’m not bashing Apple or Magnum. It’s just that the remarkable feat of Lc0 naturally leads to a discussion about what platform to run it on.

I own a (small) Mac and think Apple silicon is very impressive. Magnum does too. Where we differ is Apple’s suitability for Lc0 at the moment.

[phhnguyen]

GPU hardware is improving at a much faster rate than CPU it seems. So the future of Lc0 is looking good, apart from the huge power consumption of the high-end cards.

I wonder if TCEC has upgraded their GPU cards for the last tournament. Otherwise, the reason of LC0’s win is still not clear to me.

[Modern Times]

I wonder if TCEC has upgraded their GPU cards for the last tournament. Otherwise, the reason of LC0’s win is still not clear to me.

Clearly, it was running on a Mac

[phhnguyen]

I wonder if TCEC has upgraded their GPU cards for the last tournament. Otherwise, the reason of LC0’s win is still not clear to me.

Clearly, it was running on a Mac

Day dream!
But who knows the future!!!

[Jouni]

Didn't You notice number of games ? For equal engines we need 1000 games to get rough idea about difference.

[Ajedrecista]

Hello Jouni:

Didn't You notice number of games ? For equal engines we need 1000 games to get rough idea about difference.

Please imagine two exactly equal engines, for example SF 15.1 and Ctrl+C & Ctrl+V of the executable. Then, the density function of the trinomial model with n = 1000 games is:

Code: Select all

{W,D,L} := probability of engine SF 15.1 (the original, not the copied executable) of win, draw and lose a single game.
W + D + L = 1

{w,d,l} := number of wins, draws and loses in a n-game match.
w + d + l = n

Density function of w wins, d draws and l loses after n games:
   n!
-------- W^w D^d L^l
w! d! l!

Then, knowing that n = 1000 and also knowing that W = L (an engine against a copy of it): D = 1 - (W + L) = 1 - 2W; W = (1 - D)/2. Then, the density function of an even match (w = l) under these conditions is:

Code: Select all

      1000!
----------------- {[(1-D)/2]^(2w)} [D^(1000-2w)]
(w!²)[(1000-2w)!]

w=0,1,...,499,500

And sum to see the probability of an even match after 1000 games. I computed the probabilities with Derive 6 for some values of the expected draw ratio D:

Code: Select all

sum(vector((1000!/(((w!)^2)*((1000-2*w)!)))*(((1-D)/2)^(2*w))*(D^(1000-2*w)),w,0,500))

// Change the value of D in each computation.

Code: Select all

n = 1000 games.

  D      P(w = l)
-----------------
 0.00    0.025225
 0.05    0.012940
 0.10    0.013295
 0.15    0.013680
 0.20    0.014102
 0.25    0.014564
 0.30    0.015076
 0.35    0.015645
 0.40    0.016284
 0.45    0.017008
 0.50    0.017839
 0.55    0.018804
 0.60    0.019946
 0.65    0.021324
 0.70    0.023034
 0.75    0.025234
 0.80    0.028217
 0.85    0.032588
 0.90    0.039929
 0.95    0.056540
 1.00    1.000000

Around 1.25 seconds to 1.3 seconds to compute each value. With 1000 games against exact engines, you get less than a 5% chance of get a perfect even test (wins = loses) after 1000 games with typical draw ratios of SF 15.1 at self play (80% < D < 95%).

· Just out of curiosity, the minimum of P(w = l) with n = 1000 and W = L should be around D ~ 0.00413 and P ~ 0.0126417874 after trial and error.
· P(w = l) = 0.5 (equal chance of get an even or an uneven match after 1000 games against two exact engines) comes with D ~ 0.9991236330 (45.8 seconds to solve).

Other thing is error bars with a given confidence level and check if a difference of 0 Elo is inside or ouside error bars.

Regards from Spain.

Ajedrecista.