Tactical 'weakness' of LC0 in a recent CCCC game

[Guenther]

Interesting tactical blunder by LC0 in stage 2 of CCCC:
Here LC0 played 21...Ng6?? which lost immediately a piece due to backrank problems after 22. e5!
For chess programs normally quite easy, but from a Humans POV it looks not trivial.

[d]2r2rk1/4nppp/2b1pq2/1p6/2pRP2P/2P2BB1/3Q1PP1/R5K1 b - h3 0 21

[pgn][Event "CCCC 1: Rapid Rumble (15|5) Stage 2"] [Site "Chess.com"] [Date "2018.09.18"] [Round "?"] [White "Andscacs 0.94"] [Black "Lc0 17.11089"] [Result "1-0"] [WhiteElo "3244"] [BlackElo "3404"] [WhiteType "program"] [BlackType "program"] [ECO "D00"] [Opening "Queen's pawn, Mason variation"] [Variation "Steinitz counter-gambit"] [TimeControl "900+5"] [Time "11:27:32"] [Termination "normal"] [PlyCount "113"] 1. d4 d5 2. Bf4 c5 3. e3 Nc6 4. Nf3 Nf6 5. dxc5 e6 6. c4 Bxc5 7. Nc3 Bb4 8. Be2 dxc4 9. Qc2 Nd5 10. O-O Bxc3 11. bxc3 O-O 12. Bg3 b5 13. Rfd1 Qf6 14. Nd4 Nxd4 15. Rxd4 Bb7 16. a4 a6 17. Bf3 Rac8 18. Qd2 Bc6 19. axb5 axb5 20. e4 Ne7 21. h4 Ng6?? 22. e5 Nxe5 23. Bxe5 Qxe5 24. Bxc6 g6 25. Re1 Qc5 26. Bf3 h5 27. Rd7 Rfe8 28. Qf4 Re7 29. Red1 b4 30. Be4 bxc3 31. Qf6 Rxd7 32. Rxd7 Qf8 33. Bxg6 Qg7 34. Bxf7+ Kh8 35. Rd8+ Rxd8 36. Qxd8+ Kh7 37. Bxe6 Qe5 38. Qd7+ Qg7 39. Bf5+ Kg8 40. Qe8+ Qf8 41. Qxh5 Qg7 42. Qe8+ Qf8 43. Qxf8+ Kxf8 44. g4 Ke7 45. h5 Kf7 46. g5 Ke7 47. h6 Kf7 48. g6+ Kf8 49. g7+ Kf7 50. Bh7 Ke6 51. g8=Q+ Ke5 52. Qxc4 Kd6 53. Bc2 Ke5 54. h7 Kd6 55. h8=Q Ke7 56. Qc6 Kf7 57. Qd7# 1-0[/pgn]

[MikeGL]

Interesting tactical blunder by LC0 in stage 2 of CCCC:
Here LC0 played 21...Ng6?? which lost immediately a piece due to backrank problems after 22. e5!
For chess programs normally quite easy, but from a Humans POV it looks not trivial.

[d]2r2rk1/4nppp/2b1pq2/1p6/2pRP2P/2P2BB1/3Q1PP1/R5K1 b - h3 0 21

[pgn][Event "CCCC 1: Rapid Rumble (15|5) Stage 2"] [Site "Chess.com"] [Date "2018.09.18"] [Round "?"] [White "Andscacs 0.94"] [Black "Lc0 17.11089"] [Result "1-0"] [WhiteElo "3244"] [BlackElo "3404"] [WhiteType "program"] [BlackType "program"] [ECO "D00"] [Opening "Queen's pawn, Mason variation"] [Variation "Steinitz counter-gambit"] [TimeControl "900+5"] [Time "11:27:32"] [Termination "normal"] [PlyCount "113"] 1. d4 d5 2. Bf4 c5 3. e3 Nc6 4. Nf3 Nf6 5. dxc5 e6 6. c4 Bxc5 7. Nc3 Bb4 8. Be2 dxc4 9. Qc2 Nd5 10. O-O Bxc3 11. bxc3 O-O 12. Bg3 b5 13. Rfd1 Qf6 14. Nd4 Nxd4 15. Rxd4 Bb7 16. a4 a6 17. Bf3 Rac8 18. Qd2 Bc6 19. axb5 axb5 20. e4 Ne7 21. h4 Ng6?? 22. e5 Nxe5 23. Bxe5 Qxe5 24. Bxc6 g6 25. Re1 Qc5 26. Bf3 h5 27. Rd7 Rfe8 28. Qf4 Re7 29. Red1 b4 30. Be4 bxc3 31. Qf6 Rxd7 32. Rxd7 Qf8 33. Bxg6 Qg7 34. Bxf7+ Kh8 35. Rd8+ Rxd8 36. Qxd8+ Kh7 37. Bxe6 Qe5 38. Qd7+ Qg7 39. Bf5+ Kg8 40. Qe8+ Qf8 41. Qxh5 Qg7 42. Qe8+ Qf8 43. Qxf8+ Kxf8 44. g4 Ke7 45. h5 Kf7 46. g5 Ke7 47. h6 Kf7 48. g6+ Kf8 49. g7+ Kf7 50. Bh7 Ke6 51. g8=Q+ Ke5 52. Qxc4 Kd6 53. Bc2 Ke5 54. h7 Kd6 55. h8=Q Ke7 56. Qc6 Kf7 57. Qd7# 1-0[/pgn]

Thanks for the game. Nice tactical line in case 24...Rxc6?? of black instead of the text move 24...g6.
Good win by Andscacs.

[Daniel Shawul]

I came here to laugh at LC0's blunder but then i realized scorpio-mcts-nn on a P100 makes exactly the same mistake.
It does a 14kn/s using a 6x64 net.

Code: Select all

[18:45 dabdi@hsw223 Scorpio] > ./scorpio use_nn 1 st 40 mt 384 setboard 2r2rk1/4nppp/2b1pq2/1p6/2pRP2P/2P2BB1/3Q1PP1/R5K1 b - h3 0 21 go quit
feature done=0
ht 4194304 X 16 = 64.0 MB
eht 524288 X 8 = 8.0 MB
pht 32768 X 24 = 0.8 MB
treeht 335539200 X 40 = 12799.8 MB
processors [1]
processors [384]
EgbbProbe 4.1 by Daniel Shawul
0 egbbs loaded !      
Loading neural network ...
Loading graph on /gpu:0
Neural network loaded !	
loading_time = 6s
[st = 40000ms, mt = 40000ms , hply = 41]
63 -122 696 385  Qf6-h6 Ra1-a7
64 2 1097 20393  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-h5 Rc8-a8 Bh5xf7 Qf6xf7 Ra6xa8 Ne5-f3 g2xf3 Rf8xa8
65 29 1500 102814  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-e2 h7-h6 f2-f4 Qf6-g6 Bg3-e1 Ne5-d3 h4-h5 Qg6-f6 e4-e5 Qf6-e7 Be2xd3 c4xd3 Qd2xd3 Bc6xg2 Qd3xb5 Bg2-d5
66 36 1901 178686  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-e2 h7-h6 Qd2-e3 Ne5-g6 Rd4-d6 Ng6-e7 e4-e5 Qf6-g6 Ra6-a7 Qg6-b1 Rd6-d1
67 40 2303 252079  Ne7-g6 h4-h5 Ng6-e5 Bf3-e2 Rc8-a8 Ra1-c1 h7-h6 f2-f4 Ne5-d3 e4-e5 Qf6-e7 Rc1-f1 b5-b4 Rd4xc4 Qe7-a7 Rc4-d4 Nd3-c5 Qd2-c2 b4xc3 Rf1-d1
68 42 2704 324559  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-e2 h7-h6 Qd2-e3 Ne5-g6 h4-h5 Ng6-e7 Ra6-a7 e6-e5 Rd4-d2 Rc8-d8 Be2-f3 Ne7-c8 Ra7-a6 b5-b4 c3xb4 Rd8-d4
69 44 3108 399178  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-e2 h7-h6 Qd2-e3 Ne5-g6 h4-h5 Ng6-e7 Rd4-d6 Rf8-d8 Ra6-b6 b5-b4 e4-e5 Qf6-f5 Be2xc4 b4xc3 Rd6xd8 Rc8xd8 Qe3xc3 Qf5xh5
70 46 3509 472691  Ne7-g6 Ra1-a6 Ng6-e5 Bf3-e2 h7-h6 Qd2-e3 Ne5-g6 h4-h5 Ng6-e7 Rd4-d6 Rf8-d8 Ra6-a5 Rc8-a8 Be2-g4 Rd8-f8 e4-e5 Qf6-g5 Ra5xa8 Bc6xa8 Bg4-e2 Ne7-f5 Qe3-f4 Qg5xf4 Bg3xf4 Nf5xd6 e5xd6
71 45 3909 548048  Ne7-g6 Ra1-a6 Qf6-e7 Rd4-d6 Qe7-b7 Ra6-a1 Rc8-a8 Ra1-f1 e6-e5 Rf1-e1 b5-b4 c3xb4 f7-f6 Bf3-g4 Ng6-f4 Bg3xf4 e5xf4

#  1   -121     248 Rc8-a8 Ra1xa8 Bc6xa8 Rd4-d8 Ne7-c6 Rd8-d7
#  2   -327      81 e6-e5 Rd4-d6 Qf6-g6 Bg3xe5
#  3     55  499157 Ne7-g6 Ra1-a6 Qf6-e7 Rd4-d6 Qe7-b7 Ra6-a1 Rc8-a8 Ra1-f1 e6-e5 Rf1-e1 Rf8-e8 Re1-d1 b5-b4 h4-h5 b4xc3 Qd2-c2 Ng6-f4
#  4     -3   20054 b5-b4 Rd4xc4 b4-b3 Rc4-b4 Ne7-g6 Qd2-d4 e6-e5 Qd4-b6 Ng6xh4 Bf3-g4 Rc8-d8 Bg3xh4 b3-b2 Ra1-a5 b2-b1=R Rb4xb1 Qf6xh4 Qb6xc6 Qh4xg4 f2-f3 Qg4-f4 Qc6-c7 Rf8-e8 Qc7-c6 Re8-f8
#  5     23   23128 h7-h6 Ra1-a6 Rf8-e8 e4-e5 Qf6-g6 Qd2-d1 b5-b4 Bf3xc6 Rc8xc6 Ra6xc6 Ne7xc6 h4-h5 Qg6-g5 Rd4xc4 Re8-d8 Qd1-e1 Nc6-e7 Rc4xb4 Qg5xh5 Rb4-d4 Rd8-b8 Rd4-d7 Ne7-f5 Bg3-h2 Qh5-g4 Qe1-d1 Qg4-h4 Qd1-d3 Rb8-c8 Qd3-d2 Qh4-e4
#  6   -163      36 Bc6-b7 e4-e5 Qf6-h6 Bf3xb7 Ne7-f5
#  7    -45    6068 Rf8-e8 e4-e5 Qf6-f5 Bf3-g4 Rc8-a8 Ra1-c1 Qf5-g6 h4-h5 Qg6-h6 Bg3-f4 g7-g5 Bf4xg5 Qh6-f8 Bg5-f6 Ne7-f5 Bg4xf5
#  8    -44    6030 Rc8-e8 e4-e5 Qf6-f5 Bf3-g4 Re8-a8 Ra1-e1 Qf5-g6 h4-h5 Qg6-h6 Bg3-f4 g7-g5 Bf4xg5 Ra8-a2 Bg5xh6 Ra2xd2
#  9    -50    1868 Kg8-h8 e4-e5 Qf6-f5 Bf3-g4 Rc8-a8 Ra1-f1 Qf5-g6 h4-h5 Qg6-h6
# 10    -61     561 Qf6-g6 Bg3-d6 e6-e5 Bd6xe5 Rc8-a8 Ra1-e1 f7-f6
# 11    -51    1755 h7-h5 Bf3xh5 b5-b4 Rd4xc4 Rc8-a8 Ra1-b1 Rf8-d8 Bg3-d6
# 12    -95     149 g7-g6 e4-e5 Qf6-h8 Bf3xc6 Rc8xc6
# 13    -85     196 Bc6-e8 e4-e5 Qf6-f5 Qd2-e3 Ne7-d5
# 14    -58     803 Bc6-a8 e4-e5 Qf6-h6 Bg3-f4 Qh6-g6 h4-h5 Qg6-f5 g2-g4 Qf5xh5 g4xh5
# 15    -45    4788 Qf6-h6 Bg3-f4 Qh6-g6 h4-h5 Qg6-f6 e4-e5 Qf6-f5 g2-g4 Rc8-a8 g4xf5 Ra8xa1
# 16   -234      17 g7-g5 Qd2xg5 Qf6xg5 h4xg5 Rc8-a8 Rd4-d1
# 17   -130      75 Rc8-b8 Bg3xb8 e6-e5 Rd4-d6
# 18   -317      11 Ne7-f5 e4xf5 Bc6xf3 g2xf3
# 19   -531       8 Bc6xe4 Rd4xe4 Rf8-d8
# 20   -526       9 Ne7-d5 e4xd5 Rc8-a8 Ra1-e1
# 21   -572      15 Bc6-d5 e4xd5 Ne7xd5 Bf3xd5
# 22   -514      34 Bc6-d7 Rd4xd7 e6-e5 Ra1-d1
# 23   -632       4 Qf6xd4 c3xd4 Rc8-a8
# 24   -514       4 Rc8-c7 Bg3xc7 e6-e5 Rd4-d6
# 25   -555      23 Rc8-d8 Rd4xd8 Ne7-g6 Ra1-d1
# 26   -897      49 Qf6xf3 g2xf3 Ne7-g6 h4-h5
# 27   -689       4 Rf8-d8 Rd4xd8 Bc6-e8
# 28  -1092       6 Qf6-f5 e4xf5 Ne7xf5 Rd4-g4
# 29  -1185       3 Qf6xh4 Bg3xh4 Ne7-g6
# 30  -1252       8 Qf6-f4 Qd2xf4 Rc8-a8
# 31  -1254       3 Qf6-g5 h4xg5 Rc8-a8
# 32  -1311       3 Qf6-e5 Bg3xe5 Rc8-a8

nodes = 296910666 <75% qnodes> time = 40116ms nps = 7401302 eps = 4799764 nneps = 13865
Tree: nodes = 20391712 depth = 62 pps = 14088 visits = 565167 
      qsearch_calls = 52233 search_calls = 0
move e7g6
Bye Bye

Without using neural networks (i.e. using scorpio's handwritten eval) it does 225kn/s using 32-core cpu. It still makes the mistake though.

Code: Select all

[18:50 dabdi@hsw223 Scorpio] > ./scorpio use_nn 0 st 40 mt 32 backup_type 0 setboard 2r2rk1/4nppp/2b1pq2/1p6/2pRP2P/2P2BB1/3Q1PP1/R5K1 b - h3 0 21 go quit
feature done=0
ht 4194304 X 16 = 64.0 MB
eht 524288 X 8 = 8.0 MB
pht 32768 X 24 = 0.8 MB
treeht 335539200 X 40 = 12799.8 MB
processors [1]
processors [32]
EgbbProbe 4.1 by Daniel Shawul
0 egbbs loaded !      
loading_time = 0s
[st = 40000ms, mt = 40000ms , hply = 41]
63 11 400 401341  Ne7-g6 Rd4-d6 Bc6-a8 Ra1-a7 Ng6-e5 Bg3xe5 Qf6xe5 Qd2-d4 Qe5xd4 Rd6xd4 Rf8-d8 Rd4xd8 Rc8xd8
64 34 800 840046  Ne7-g6 Ra1-a7 Ng6xh4 Bg3xh4 Qf6xh4 e4-e5 Qh4-h6 Qd2xh6 g7xh6 Bf3xc6 Rc8xc6 Ra7-b7 Rc6-c8 Rb7xb5 Rf8-d8 Rd4-d6 Rc8-a8 Rb5-b4 Rd8-c8 Rb4-b7 Ra8-a1 Kg1-h2
65 58 1200 1285393  Ne7-g6 Rd4-d6 Bc6-a8 Qd2-d4 Qf6xd4 Rd6xd4 e6-e5 Rd4-d7 Rf8-d8 Rd7-a7 Ba8-c6 h4-h5 Ng6-f4 Bf3-g4 Rc8-b8 Ra7-e7 b5-b4 Re7xe5
66 24 1600 1974801  Ne7-g6 Rd4-d6 Ng6-e5 Bf3-e2 h7-h5 Qd2-g5 Qf6xg5 h4xg5 Ne5-d3 f2-f3 Rf8-d8 e4-e5 Bc6-d5 f3-f4 b5-b4 c3xb4
67 -2 2000 2642212  Ne7-g6 Rd4-d6 Ng6-e5 Bf3-e2 Ne5-g6 Be2-f3
68 -2 2400 3467002  Ne7-g6 Rd4-d6 Ng6-e5 Bf3-e2 Ne5-g6 Be2-f3
69 -2 2800 4546923  Ne7-g6 Rd4-d6 Ng6-e5 Bf3-e2 Ne5-g6 Be2-f3
70 -2 3201 5638815  Ne7-g6 Rd4-d6 Ng6-e5 Bf3-e2 Ne5-d3 Be2xd3 c4xd3 Qd2xd3 Qf6-g6 Ra1-e1 Bc6-b7 Rd6-d7 Bb7-c6 Rd7-d6
71 -1 3601 6964375  Ne7-g6 Rd4-d6 Bc6-e8 h4-h5 Ng6-e5 Bf3-e2 b5-b4 Qd2-d4 Ne5-d3 Qd4xf6 g7xf6 c3xb4 Be8-b5 f2-f3 Rc8-b8 Bg3-h4 Kg8-g7 h5-h6 Kg7-g6 Bh4-f2 Kg6xh6

#  1   -152    3046 Rc8-a8 Ra1xa8 Bc6xa8 Rd4-d8 Ba8-c6 Qd2-d6 e6-e5 Rd8xf8 Kg8xf8 Bg3xe5 Qf6xd6 Be5xd6 f7-f6
#  2   -720       9 e6-e5 Rd4-d6 Rc8-a8
#  3     -2 8412844 Ne7-g6 Rd4-d6 b5-b4 e4-e5 Ng6xe5 Bg3xe5 Qf6xe5 Rd6xc6 Rc8xc6 Bf3xc6 b4-b3 Qd2-b2 Qe5-c5 Bc6-f3 Rf8-d8 Ra1-f1 Rd8-d3 Rf1-a1 Rd3-d8
#  4   -129   12873 b5-b4 Rd4xc4 b4xc3 Qd2xc3 Qf6xc3 Rc4xc3 Rf8-d8 Ra1-a7 Bc6xe4 Ra7xe7 Rc8xc3 Bf3xe4 Rc3-c4 Be4-f3 Rc4-c1 Kg1-h2 Rc1-b1 Bg3-e5
#  5   -104  507006 h7-h6 h4-h5 b5-b4 Rd4xc4 b4xc3 Qd2xc3 Qf6xc3 Rc4xc3 Rf8-d8 Ra1-a7 Bc6xe4 Ra7xe7 Rc8xc3 Bf3xe4 Rc3-c5 Be4-f3 Rc5-c1 Kg1-h2 Rc1-c2 Re7-c7 Rc2xc7
#  6   -566      10 Bc6-b7 e4-e5 Qf6-g6
#  7   -206   41286 Rf8-e8 e4-e5 Qf6-f5 Bf3-g4 Rc8-a8 Ra1-d1 Qf5-g6 h4-h5 Qg6-h6 Bg3-f4 g7-g5
#  8   -144    2393 Rc8-e8 Rd4-d6 Re8-c8 Ra1-a7 b5-b4 e4-e5 Qf6-h6 Qd2xh6 g7xh6 Ra7xe7 Bc6xf3 g2xf3 b4xc3 Rd6-d7
#  9   -126   17725 Kg8-h8 h4-h5 h7-h6 Ra1-a7 Ne7-g8 Bg3-c7 Qf6-e7 Bc7-b6 Qe7-h4 Bb6-c5 e6-e5
# 10   -192     533 Qf6-g6 Ra1-a7 Rc8-e8 Bg3-d6 e6-e5 Bd6xe5
# 11   -156    5792 h7-h5 Ra1-a7 Ne7-g6 Bf3xh5 Ng6-e5 f2-f3
# 12   -144    1141 g7-g6 h4-h5 Qf6-h8 Ra1-a7 Rf8-e8 h5xg6
# 13   -155    1091 Bc6-e8 e4-e5 Qf6-g6 h4-h5 Qg6-f5 h5-h6 g7-g6 Bg3-h4
# 14   -182    1392 Bc6-a8 Rd4-d7 b5-b4 Qd2-g5 Qf6xg5 h4xg5 Ne7-c6 c3xb4 c4-c3 b4-b5
# 15   -388      51 Qf6-h6 Bg3-d6 Qh6xd2
# 16   -320      33 g7-g5 h4xg5 Qf6-h8 Ra1-a7 e6-e5
# 17   -691       9 Rc8-b8 Bg3xb8 Bc6xe4
# 18   -293      29 Ne7-f5 e4xf5 Bc6xf3 g2xf3 Qf6xf5
# 19   -500      11 Bc6xe4 Bf3xe4 Ne7-f5
# 20   -473      12 Ne7-d5 e4xd5 e6xd5
# 21   -490      12 Bc6-d5 e4xd5 e6xd5
# 22   -689       9 Bc6-d7 Rd4xd7 b5-b4
# 23   -777       9 Qf6xd4 Qd2xd4 Rc8-a8
# 24   -749       9 Rc8-c7 Bg3xc7 Qf6xh4
# 25   -850       9 Rc8-d8 Rd4xd8 b5-b4
# 26   -962       9 Qf6xf3 g2xf3 Rc8-a8
# 27   -952       9 Rf8-d8 Rd4xd8 Bc6-e8 e4-e5
# 28  -1246       8 Qf6-f5 e4xf5 Ne7xf5 Rd4-f4
# 29  -1484       8 Qf6xh4 Bg3xh4 f7-f6 Rd4-d6
# 30  -1596       8 Qf6-f4 Qd2xf4 Rc8-a8
# 31  -1641       8 Qf6-g5 h4xg5 Ne7-g6 Bg3-d6
# 32  -1625       8 Qf6-e5 Bg3xe5 f7-f6 Be5-d6

nodes = 1107337102 <88% qnodes> time = 40002ms nps = 27682042 eps = 20091206 nneps = 0
Tree: nodes = 131555716 depth = 43 pps = 225172 visits = 9007361 
      qsearch_calls = 4055776 search_calls = 0
move e7g6
Bye Bye

[MikeGL]

Maybe intentional as the resulting position is 2 pawns versus B of white, but c-pawn is also a dangerous passer for black.
Hence, the possible compensation. But not enough to win for black, not even a draw.

[AndyKHC]

It's not a weakness that is replicable across all the NNs. I looked at 4 other NNs which have done well in testing for me.

Of these both 11262 and 11187 prefer 21. .. h6 without considering Ng6 which was played by 11089. 11120 (which is close-ish to 11089) starts with Ng6 but switches to h6 very quickly (depth 11 if I recall).

More interestingly, 11066 which is an earlier net compared to 11089 also starts with Ng6 but crucially switches to h6 like 11120 shortly after as well. I left 11089 running and once it settles on Ng6, it doesn't consider anything else after running for a good 15 minutes on my GTX 1070.

[jp]

It's not a weakness that is replicable across all the NNs. I looked at 4 other NNs which have done well in testing for me.

Do they all agree it's bad & find the PV once it's been played?