Question to Lc0 experts

Eduard · Post by **Eduard** » Thu Apr 28, 2022 9:08 am

Question to Lc0 experts

I'm working on an opening book for Lc0, and Lc0 plays great with it. Unfortunately, some strange moves happened recently, that I can't explain and it's also very difficult to improve in the book. I can't explain the moves. What can be the reason that Lc0 made these gross mistakes? Lc0 is playing with a T 78 Network on a RTX 3060m.

Position 1:

[fen]5rk1/ppr1bpp1/2n4p/4N3/2R4P/q3P3/2Q1BPP1/1R4K1 b - - 0 1[/fen]

Analysis by Blue Marlin 15.1:

24...Rfc8 25.Bf3 Qa5 26.Nxc6 bxc6 27.h5 Rd8 28.g3 Qd2 29.Qa4 Qd3 30.Rd1 Qf5 31.Rxd8+ Bxd8 32.Rf4 Qd7 33.Bg4 Qe8 34.Bd1 Rb7 35.Bf3 Rc7 36.Rf5 Bf6 37.Be2 Qb8 38.Qg4 Qd8 39.Bc4 Kf8 40.Qe4 Re7 41.Qf4 Re8 42.Rc5 Qd7
White is slightly better: +/= (0.51) Depth: 33/41 00:00:08 100MN, tb=297

And what did Lc0 play? 24...Bxh4? (Played in Depth 12 "Bxh4 {0.26/12 5").

Game:

[Event "Wertungspartie, 5 min"]
[Site "Maschinen A"]
[Date "2022.04.23"]
[Round "?"]
[White "Crikoch, EMAN 8.01 64-bit AV"]
[Black "Python, Lc0 v0.29.0-rc0"]
[Result "1-0"]
[ECO "D37"]
[WhiteElo "2798"]
[BlackElo "2815"]
[Annotator "3.80;0.16"]
[PlyCount "135"]
[EventDate "2022.03.31"]
[TimeControl "300"]

{EMAN 8.01 64-bit AVX2 (126 threads): 50.8 plies; 106.558kN/s AMD Ryzen
Threadripper PRO 3995WX 64-Cores 2695MHz, (64 cores, 128 threads),
Python230422.ctg, 1024 MB} 1. Nf3 {0.00/0 0} Nf6 {0.00/0 0} 2. d4 {0.00/0 0} e6
{0.00/0 0} 3. c4 {0.00/0 0} d5 {0.00/0 0} 4. Nc3 {0.00/0 0} Be7 {0.00/0 0} 5.
Bf4 {0.00/0 0} O-O {0.00/0 0} 6. e3 {0.00/0 0} c5 {0.00/0 0} 7. dxc5 {0.00/0 0}
Bxc5 {0.00/0 0} 8. a3 {0.00/0 0} Nc6 {0.00/0 0} 9. Qc2 {0.00/0 0} Qa5 {0.00/0 0
} 10. Rd1 {0.00/0 0} Be7 {0.00/0 0} 11. Be2 {0.00/0 0} Ne4 {0.00/0 0} 12. cxd5
{0.00/0 0} Nxc3 {0.00/0 0} 13. bxc3 {0.00/0 0} exd5 {0.00/0 0} 14. O-O {
0.00/0 0} Bf6 {0.00/0 0} 15. c4 {0.00/0 0} Be6 {0.00/0 0} 16. cxd5 {0.00/0 0}
Bxd5 {0.00/0 0} 17. h4 {0.00/0 0} h6 {0.00/0 0} 18. Bc7 {0.00/0 0} Qxc7 {
0.00/0 0} 19. Rxd5 {0.00/0 0} Rac8 {0.16/8 12 [%wdl 235,646,119]} 20. Rc5 {
0.00/0 0 (Db1)} Be7 {0.13/11 7 [%wdl 245,604,151]} 21. Rc3 {0.00/0 0} Qa5 {
0.11/13 0 [%wdl 229,623,148]} 22. Rc4 {0.00/0 0} Qxa3 {0.11/12 5 [%wdl 216,651,
133]} 23. Rb1 {0.00/0 0} Rc7 {0.11/13 5 [%wdl 197,685,118]} 24. Ne5 {0.00/0 0}
Bxh4 {0.26/12 5 [%wdl 274,634,92]} 25. Rxh4 {3.80/38 17 (Sxc6)} Qe7 {2.00/15 5
[%wdl 802,132,66]} 26. Re4 {3.85/40 6} Nxe5 {2.60/15 0 [%wdl 834,123,43]} 27.
Qb2 {4.21/41 0} f6 {2.72/15 5 [%wdl 837, 127,36] (Te8)} 28. f4 {4.43/36 7} Qc5
{2.72/16 5 [%wdl 835,130,35]} 29. fxe5 {4.56/36 5} f5 {2.74/15 5 [%wdl 834,134,
32]} 30. Qb3+ {4.51/35 1} Kh8 {2.94/14 0 [%wdl 844,127,29] (Kh7)} 31. Rc4 {
4.93/39 17} Qxe5 {3.14/15 5 [%wdl 851,124,25]} 32. Rxc7 {5.12/33 13} Qxc7 {
3.29/14 5 [%wdl 858,119,23]} 33. Qa3 {5.17/36 3 (Dxb7)} Ra8 {3.47/11 16 [%wdl
863,116,21] (Td8)} 34. Bf3 {5.42/28 12} Qg3 {3.79/10 11 [%wdl 875, 105,20]} 35.
Qc3 {5.95/35 9 (De7)} Rd8 {3.38/9 5 [%wdl 865,108,27]} 36. Rxb7 {6.52/30 1} Rb8
{5.38/12 5 [%wdl 919,60,21] (a6)} 37. Bg4 {20.28/25 7 (Dxg7+)} Rxb7 {6.54/14 5
[%wdl 935,47,18] (Tg8)} 38. Qc8+ {100.00/152 6} Kh7 {7.82/1 0 [%wdl 944,43,13]}
39. Bxf5+ {100.00/55 0} g6 {8.61/14 5 [%wdl 947,43,10]} 40. Qxb7+ {100.00/57 1}
Kh8 {9.40/13 5 [%wdl 951,42,7]} 41. Qa8+ {100.00/53 44} Kg7 {12.21/12 5 [%wdl
964,30,6]} 42. Qxa7+ {#73/51 22} Kf6 {12.16/0 5 [%wdl 964,30,6]} 43. Bh3 {
#67/52 0} h5 {12.69/9 5 [%wdl 965,29,6]} 44. Qd4+ {#61/50 2} Kg5 {11.99/9 14
[%wdl 962,33,5]} 45. Bd7 {#57/57 0 (Df4+)} Qc7 {12.56/7 5 [%wdl 964,31,5]
(De1+)} 46. Kf2 {#49/49 7 (Lb5)} Qc2+ {12.04/8 5 [%wdl 962,33,5]} 47. Kg3 {
#47/59 0 (Kf3)} Kh6 {11.76/8 5 [%wdl 962,32,6]} 48. Qh8+ {#45/61 1 (Le6)} Kg5 {
10.58/1 0 [%wdl 957,37,6]} 49. Qf8 {#43/66 5 (De5+)} Qc7+ {#253/1 0 [%wdl 1000,
0,0] (h4+)} 50. Qf4+ {#37/67 3} Qxf4+ {#251/1 0 [%wdl 1000,0,0]} 51. exf4+ {
#35/65 3} Kh6 {#249/1 0 [%wdl 1000,0,0] (Kf6)} 52. Kh4 {#33/61 4 (Lg4)} g5+ {
#241/1 0 [%wdl 1000,0,0] (Kg7)} 53. fxg5+ {#25/0 3} Kg6 {279.99/0 0 (Kg7)} 54.
g3 {279.99/0 0} Kf7 {279.99/0 0} 55. Kxh5 {279.99/0 0} Kg7 {279.99/0 0} 56. g4
{279.99/0 0} Kf7 {279.99/0 0} 57. g6+ {279.99/1 0} Kg7 {279.99/0 0} 58. g5 {
279.99/1 0} Kf8 {279.99/0 0} 59. g7+ {279.99/1 0} Kxg7 {279.99/0 0} 60. g6 {
279.98/1 0} Kf6 {279.98/0 0} 61. Kh6 {279.99/1 0} Ke7 {279.99/0 0} 62. g7 {
279.98/1 0} Kf7 {279.98/1 0} 63. Bc6 {279.99/0 0} Ke6 {279.99/0 0} 64. g8=Q+ {
279.96/1 0} Kf5 {279.96/0 0} 65. Bf3 {279.97/0 0} Ke5 {279.97/0 0} 66. Qd8 {
279.98/1 0} Ke6 {279.98/0 0} 67. Kg6 {279.99/0 0} Ke5 {279.99/1 0} 68. Qf6# {
#1/1 0} 1-0

Position 2:

[fen]r3r1k1/3q1pb1/1p1p2p1/pNnPp2n/2P4p/1P2BP2/P2QB1PP/4RR1K b - - 0 1[/fen]

Analysis by Blue Marlin 15.1:

21...f5 22.Kg1 Nf4 23.Kh1 Bf6 24.Bxf4 exf4 25.Qxf4 Re5 26.Qh6 Rae8 27.Qxg6+ Qg7 28.Qxg7+ Bxg7 29.Nxd6 R8e7 30.Nc8 Re8
The position is equal: = (0.00) Depth: 43/19 00:00:16 209MN

The position is equal, but Lc0 now plays 21...Ng3+? (Played in Depth 26 "Ng3+ -0.07/26 8")

Mating is fine, but if there's no mate, then at a level where you're playing against Stockfish (!), such moves are crap!

Game:

[Event "Wertungspartie, 5 min"]
[Site "Maschinenraum"]
[Date "2022.04.27"]
[Round "?"]
[White "TicTacto, EMAN 8.01 64-bit BM"]
[Black "Python, Lc0 v0.30.0-dev+git"]
[Result "1-0"]
[ECO "B90"]
[WhiteElo "2812"]
[BlackElo "2805"]
[Annotator "0.00;0.01"]
[PlyCount "65"]
[EventDate "2022.02.26"]
[TimeControl "300"]

{Python270422.ctg, 512 MB} 1. e4 {0.00/0 0} c5 {0.00/0 0} 2. Nf3 {0.00/0 0} d6
{0.00/0 0} 3. d4 {0.00/0 0} cxd4 {0.00/0 0} 4. Nxd4 {0.00/0 0} Nf6 {0.00/0 0}
5. Nc3 {0.00/0 0} a6 {0.00/0 0} 6. Be3 {0.00/0 0} e5 {0.00/0 0} 7. Nb3 {
0.00/0 0} Be6 {0.00/0 0} 8. f3 {0.00/0 0} h5 {0.00/0 0} 9. Qd2 {0.00/0 0} Nbd7
{0.00/0 0} 10. Nd5 {0.00/0 0} Bxd5 {0.00/0 0} 11. exd5 {0.00/0 0} g6 {0.00/0 0}
12. Be2 {0.00/0 0} Bg7 {0.00/0 0} 13. O-O {0.00/0 0} b6 {0.00/0 0} 14. c4 {
0.00/0 0} O-O {0.00/0 0} 15. Rae1 {0.00/0 0} Re8 {0.00/0 0} 16. Kh1 {0.01/0 0}
a5 {0.00/0 0} 17. Na1 {0.01/0 0} Nc5 {0.00/0 0} 18. Nc2 {0.01/0 0} Qd7 {
0.00/0 0} 19. b3 {0.01/0 0} h4 {0.01/15 8 [%wdl 288,438,274]} 20. Na3 {0.01/0 0
} Nh5 {0.04/17 5 [%wdl 298,438,264]} 21. Nb5 {0.00/0 23 (Ld1)} Ng3+ {-0.07/26
8 [%wdl 228,489,283] (Sf4)} 22. hxg3 {2.34/31 21} hxg3 {-0.05/31 5 [%wdl 211,
536,253]} 23. Kg1 {2.85/28 1 (Lg5)} f5 {-0.79/17 8 [%wdl 106,326,568]} 24. Bxc5
{2.55/31 0 (Ld1)} Qd8 {0.36/25 14 [%wdl 486,276,238] (bxc5)} 25. Bf2 {3.81/27
18} f4 {0.54/28 5 [%wdl 533, 283,184]} 26. Rc1 {3.64/24 0} Qh4 {0.58/30 5
[%wdl 538,293,169]} 27. Rfd1 {4.06/25 0} e4 {0.59/28 5 [%wdl 536,300,164]
(gxf2+)} 28. fxe4 {4.25/27 15} Rxe4 {0.59/26 0 [%wdl 532,311,157] (gxf2+)} 29.
Bf3 {5.34/23 7 (Kf1)} Rae8 {2.21/0 10 [%wdl 812,134,54] (Te7)} 30. Nxd6 {
8.79/21 0} Re2 {6.52/12 8 [%wdl 940,37,23] (Lc3)} 31. Bxe2 {10.27/24 5} gxf2+ {
11.78/13 0 [%wdl 965,26,9]} 32. Kf1 {10.89/26 0} Re3 {11.48/11 8 [%wdl 963,28,
9] (Dh2)} 33. Bf3 {11.73/21 7 Python, Lc0 v0.30.0-dev+git. gibt auf} 1-0

Jouni · Post by **Jouni** » Thu Apr 28, 2022 11:09 am

I am not expert. But I see that even old Lc0 0.27.0 in GTX card wants to play same moves! Lc0 simply blunders occasionally. Happen also in TCEC subfinal and super hardware.

AdminX · Post by **AdminX** » Thu Apr 28, 2022 1:31 pm

Interesting, ChessAra, and Ceres prefer Qd6. I would have thought Ceres, since it is more in line with LC0 might have made the same error. I am using a 2070S over here.

Code: Select all

1:36: +0.33/32 1...Qd6 2.Nxc6 bxc6 3.g3 Bf6 4.Rd1 Qe5 5.Bf3 Qb2 6.Qf5 Qe5 7.Qc2 Qb2 8.Qa4 c5 9.Rd5 Be7 10.Rd7 Rxd7 11.Qxd7 Qf6 12.Kg2 a5 13.Re4 Bd6 14.Re8 Rxe8 15.Qxe8+ Bf8 16.Bd5 Qe7 17.Qa8 - ChessAra 0.99

1:16: +0.22/16 1...Qd6 2.Nxc6 bxc6 3.g3 Bf6 4.h5 c5 5.Rd1 Qe5 6.Bf3 Qb2 7.Qa4 Qb6 8.Kg2 Qe6 - Ceres 0.96

1:30: +0.20/15 1...Bxh4 2.Nxc6 Rxc6 3.Rxb7 Qa1+ 4.Rb1 Rxc4 5.Bxc4 Qe5 6.Rb7 a5 7.Qa2 Be7 8.g3 g6 9.Ra7 Kg7 10.Rxa5 Qf6 11.Ra6 Qe5 12.Ra7 - LC0 0.29

The same applied to position 2, ChessAra and Ceres preferred ... Nf4, only LC0 ... Ng3

Eduard · Post by **Eduard** » Thu Apr 28, 2022 3:33 pm

The balance of Lc0 with RTX 3060m on PlayChess is currently +1 - 1 =384 (stored last 386 games).

That's a very good result for a mobile GPU and Blitzgames. I am passionate about it and have been investing a lot of time in the opening book for the last 2 months. But no book can help against the mistakes shown, that makes me sad, and such stupid mistakes annoy me. We're not talking about coffeehouse chess, we're talking about 3500 Elo. But the mistakes are very stupid. When will Lc0 finally learn tactically?

Pedro · Post by **Pedro** » Thu Apr 28, 2022 10:01 pm

I have a few questions: what is Leela's strongest network right now? T60 or T70? What is the main race network now?

Last question, Komodo Dragon 2.6 is about 63 elo stronger than Leela's best network according to S.Pohl's website, however Leela is winning the match against Komodo Dragon at Tcec. What does this mean? That Leela is stronger than KomodoDragon and that Pohl's website is wrong in its assessment?

Jouni · Post by **Jouni** » Fri Apr 29, 2022 8:59 am

In TCEC Lc0 is beating Dragon 34-30, but in CCC Dragon is leading 49-43. Who is stronger

. I quess in normal hardware Dragon is better.

Eduard · Post by **Eduard** » Fri Apr 29, 2022 9:18 am

Pedro wrote: ↑Thu Apr 28, 2022 10:01 pm I have a few questions: what is Leela's strongest network right now? T60 or T70? What is the main race network now?

Last question, Komodo Dragon 2.6 is about 63 elo stronger than Leela's best network according to S.Pohl's website, however Leela is winning the match against Komodo Dragon at Tcec. What does this mean? That Leela is stronger than KomodoDragon and that Pohl's website is wrong in its assessment?

I haven't seen Dragon win tournaments on PlayChess or InfinityChess.

Lc0 does. On a good day and with a well adjusted opening book, you always have a chance with Lc0 to beat Stockfish as well (I didn't see Dragon defeat Stockfish).

50% points on PlayChess with mobile RTX 3060m (laptop) and T78 network is a great result.

Ozymandias · Post by **Ozymandias** » Fri Apr 29, 2022 9:30 am

AdminX wrote: ↑Thu Apr 28, 2022 1:31 pm Interesting, ChessAra, and Ceres prefer Qd6. I would have thought Ceres, since it is more in line with LC0 might have made the same error. I am using a 2070S over here.
Code: Select all
1:36: +0.33/32 1...Qd6 2.Nxc6 bxc6 3.g3 Bf6 4.Rd1 Qe5 5.Bf3 Qb2 6.Qf5 Qe5 7.Qc2 Qb2 8.Qa4 c5 9.Rd5 Be7 10.Rd7 Rxd7 11.Qxd7 Qf6 12.Kg2 a5 13.Re4 Bd6 14.Re8 Rxe8 15.Qxe8+ Bf8 16.Bd5 Qe7 17.Qa8 - ChessAra 0.99

1:16: +0.22/16 1...Qd6 2.Nxc6 bxc6 3.g3 Bf6 4.h5 c5 5.Rd1 Qe5 6.Bf3 Qb2 7.Qa4 Qb6 8.Kg2 Qe6 - Ceres 0.96

1:30: +0.20/15 1...Bxh4 2.Nxc6 Rxc6 3.Rxb7 Qa1+ 4.Rb1 Rxc4 5.Bxc4 Qe5 6.Rb7 a5 7.Qa2 Be7 8.g3 g6 9.Ra7 Kg7 10.Rxa5 Qf6 11.Ra6 Qe5 12.Ra7 - LC0 0.29
The same applied to position 2, ChessAra and Ceres preferred ... Nf4, only LC0 ... Ng3

Sounds like the problem might lay with MCTS. It would be good if Ceres improvements superseded what we have as the de facto lc0 search method. But for that, it needs to be better than current lc0 overall, and it still isn't.

Pedro · Post by **Pedro** » Fri Apr 29, 2022 4:44 pm

Jouni wrote: ↑Fri Apr 29, 2022 8:59 am In TCEC Lc0 is beating Dragon 34-30, but in CCC Dragon is leading 49-43. Who is stronger . I quess in normal hardware Dragon is better.

Will the longer time control interfere with Leela's strength?

In this case, the ccc hardware is extremely powerful. I think it's far from normal.

Here:

CPUs: 2 x AMD EPYC 7H12
GPU: 2x A100 (40 GB GPU memory)
Cores: 256 cores (128 physical)
RAM: 512GB DIMM DDR4 2933 MHz (0.3 ns)
SSD: 2x Micron 5210 MTFD (2TB) in RAID1
OS: CentOS 8

Question to Lc0 experts

Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts

Re: Question to Lc0 experts