LCZero blunders in TCEC cup 4

[marsell]

This is the GPU setup:
GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16 GB DDR3-2133
SSD:Samsung 840 Pro 256 GB

It's not that bad! Is it

[George Tsavdaris]

LC0 doesn't choose Qc6 either for my setup and the score difference is pretty big (0.76).

Yes, indeed. What was odd was that my Setup didn't even look ONCE at Qc6 !
Assuming that TCEC has at least half-decent GPUs, this is quite surprising.

I have tried more than 10 times with the parameters TCEC uses and with default parameters and with close to TCEC hardware, but not even once i was able to reproduce the Qc6?? big blunder of Leela in the CUP. It was not even thinking about it!!

Moreover in all but one of the runs Leela was choosing from the beginning till the end(1 nodes till 10 million that i was stopping search) the correct Rcd4 that would draw. Only once she has chosen an alternative move(that i don't remember which) and stick with it, that still does not lose so easily like Qc6.

Strange thing. But it's not a TCEC conspiracy against Leela as you claim, these strange things happen some times....

[Zenmastur]

LC0 doesn't choose Qc6 either for my setup and the score difference is pretty big (0.76).

Yes, indeed. What was odd was that my Setup didn't even look ONCE at Qc6 !
Assuming that TCEC has at least half-decent GPUs, this is quite surprising.
Makes one wonder if the TCEC people are biased against NN Engines, especially if they are old men.
I remember that the Golden Oldies here were the most upset when AlphaZero first appeared on the scene and made mincemeat of the God of AB Engines, Stockfish. They were extremely relieved when Google showed no further interest in Computer Chess.
And there's no one happier than them that Leela's Progress has hit a temporary Roadblock.

I think you need to put that crack pipe down and go outside and get some fresh air.

Regards,

Zenmastur

[Jouni]

I just read from one forum: "There's an unknown but probably significant extent to which neural networks are memorizing an opening book." True?!

[Zenmastur]

I just read from one forum: "There's an unknown but probably significant extent to which neural networks are memorizing an opening book." True?!

One of the problems with NN is you have no way to know what they have learned! It's a black box.

Regards,

Zenmastur

[jhellis3]

It depends on what input features the NNs have access to and how they are trained. For A0, it was certainly possible that such an effect occurred (due to the presence of movecount). For LC0, the likelihood & potential impact is much less as it does not have movecount as an input. It does still have 8 ply of history (and actually uses pre-root history) with unique & consistent identifiers, and very heavily trained states/positions, so it is very likely some influence exists for at least 8 ply.

[shrapnel]

This is the GPU setup:
GPU Server
GPUs: 1 x 2080 ti + 1 x 2080
CPU: Quad Core i5 2600k
RAM: 16 GB DDR3-2133
SSD:Samsung 840 Pro 256 GB

It's not that bad! Is it

Not bad for a Private setup, but piss-poor for such a reputed Tournament.
What are their CPU Specs ? I'm sure they're impressive by any standards, so Stockfish starts with an advantage before even a Game is played.
If they had two Titan Vs ( which even I can't afford) running together, there would be a different Tale to tell for sure.
Also, 16 GB DDR3 RAM is ridiculously low. They should have at least 128 GB DDR4 RAM as Leela is a RAM-HOG and quickly occupies all the RAM that is available to it.
You just confirmed my suspicions about the abysmal conditions poor lc0 is playing under.

[shrapnel]

Strange thing. But it's not a TCEC conspiracy against Leela as you claim, these strange things happen some times....

You are the Third person in this Thread whose lc0 never even looks at Qc6.
Once maybe a coincidence, but THREE times....a little more than just Strange.
Deliberate or not I can't say, but something's wrong with the conditions under which lc0 is playing in TCEC, THAT's for sure !
You can take it to the Bank.

[MikeB]

New game
5k2/3n1p1p/7P/1pqN2P1/2r1rP2/5Q2/6K1/3R1R2 b - - 0 1

Analysis by Lc0 v0.22.0:

40...b4 41.Rfe1 Rxe1 42.Rxe1 b3 43.Kh3 Rc1 44.Rxc1 Qxc1 45.Qxb3 Qf1+ 46.Kh2 Qf2+ 47.Kh3 Qf1+ 48.Kh4 Qe1+ 49.Kg4 Qe2+ 50.Kg3 Qe1+ 51.Kf3 Qf1+ 52.Kg4 Qe2+ 53.Qf3 Qxf3+ 54.Kxf3 f6 55.Kg4 fxg5 56.Kxg5 Kf7 57.Kf5 Nf8 58.Nf6 Ke7 59.Ne4 Nd7 60.Ng5 Nf6 61.Nxh7 Nxh7 62.Kg6 Nf8+ 63.Kg7
+/= (0.54) Depth: 24/58 00:02:28 3277kN


I cannot get my LCO to play that silly move.

b4 is the best move here ...and it would still be in the game , game over after Qc6...

[Zenmastur]

New game
5k2/3n1p1p/7P/1pqN2P1/2r1rP2/5Q2/6K1/3R1R2 b - - 0 1

Analysis by Lc0 v0.22.0:

40...b4 41.Rfe1 Rxe1 42.Rxe1 b3 43.Kh3 Rc1 44.Rxc1 Qxc1 45.Qxb3 Qf1+ 46.Kh2 Qf2+ 47.Kh3 Qf1+ 48.Kh4 Qe1+ 49.Kg4 Qe2+ 50.Kg3 Qe1+ 51.Kf3 Qf1+ 52.Kg4 Qe2+ 53.Qf3 Qxf3+ 54.Kxf3 f6 55.Kg4 fxg5 56.Kxg5 Kf7 57.Kf5 Nf8 58.Nf6 Ke7 59.Ne4 Nd7 60.Ng5 Nf6 61.Nxh7 Nxh7 62.Kg6 Nf8+ 63.Kg7
+/= (0.54) Depth: 24/58 00:02:28 3277kN


I cannot get my LCO to play that silly move.

b4 is the best move here ...and it would still be in the game , game over after Qc6...

Code: Select all

(  [Stockfish 021119 64 BMI2] 58:+0.00 1...b4 2.Rfe1 Rxe1 3.Rxe1 b3 4.Kh3 Rc1 5.Re2 Qc8 6.Kg3 Rc2 7.Re1 Rc1 8.Rxc1 Qxc1 9.Qxb3 Qe1+ 10.Kg4 Qe2+ 11.Kh3 Qf1+ 12.Kh4 Qe1+ 13.Kg4 Qe2+ 14.Kg3 Qe1+ 15.Kf3 Qf1+ 16.Kg4 Qe2+ 17.Qf3 Qxf3+ 18.Kxf3 f6 19.Kg4 fxg5 20.Kxg5 Kf7 21.Kf5 Nf8 22.Nf6 Ke7 23.Ne4 Nd7 24.Ng5 Nf6 25.Nxh7 Nxh7 )
(  [Stockfish 021119 64 BMI2] 58:+0.00 1...Rcd4 2.Rxd4 Rxd4 3.Nc3 Nb6 4.Re1 Nd5 5.Ne4 Qc2+ 6.Kg1 Ra4 7.f5 Ra2 8.Nf2 Qd2 9.Kf1 Qf4 10.Qxf4 Nxf4 11.Ng4 Nh3 12.Re5 Nxg5 13.Nf6 Ra8 14.Nd7+ Kg8 15.f6 Ne6 16.Rxb5 Nf8 17.Rg5+ Ng6 18.Rd5 Nf8 19.Nb6 Rb8 20.Rd6 Re8 21.Nd5 Ne6 22.Ne7+ Kf8 23.Nf5 Ra8 24.Kf2 Ra2+ 25.Ke3 Ra3+ 26.Ke2 Ra2+ )

I guess ...b4 isn't the only best move.

Regards,

Zenmastur