Nobody's perfect - not even Stockfish

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Jouni
Posts: 3227
Joined: Wed Mar 08, 2006 8:15 pm

Nobody's perfect - not even Stockfish

Post by Jouni »

Latest dev version sometimes misses very old and easy moves. Like here
[fen]1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - 0 1[/fen]

Analysis by Stockfish 110922 1 core:

1.Qe7 Ng8 2.Qe5 Nf6 3.Kf1 Kh7 4.Qe7 Ng8 5.Qe4+ Kh8 6.Rd3 Nf6 7.Qe7 b5 8.axb5 axb5 9.Rg3 Rf8 10.Ra3 Qd8 11.Qd6 Kg8 12.Ra8 Qxa8 13.Bxa8 Nxd7 14.Be4 Nf6 15.Bc2 b4 16.Bb3 Nxh5 17.Qc7 Nf6 18.Kg2 Nh5 19.f5 Nf6 20.Qe7 Nh5 21.Kf1 Nf6 22.Ba4 Nh5 23.Kg1 Nf6 24.Qb7 Nh5 25.Bb3 Nf6 26.Qb5 Re8 27.Qxb4 Rd8 28.Qe7 Rf8 29.Kf1 Nh7 30.Qb4 Re8
+- (6.26) Depth: 47/65 00:04:49 372mN, tb=144909
Jouni
CornfedForever
Posts: 597
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Nobody's perfect - not even Stockfish

Post by CornfedForever »

Jouni wrote: Thu Sep 15, 2022 4:58 pm Latest dev version sometimes misses very old and easy moves. Like here
[fen]1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - 0 1[/fen]

Analysis by Stockfish 110922 1 core:

1.Qe7 Ng8 2.Qe5 Nf6 3.Kf1 Kh7 4.Qe7 Ng8 5.Qe4+ Kh8 6.Rd3 Nf6 7.Qe7 b5 8.axb5 axb5 9.Rg3 Rf8 10.Ra3 Qd8 11.Qd6 Kg8 12.Ra8 Qxa8 13.Bxa8 Nxd7 14.Be4 Nf6 15.Bc2 b4 16.Bb3 Nxh5 17.Qc7 Nf6 18.Kg2 Nh5 19.f5 Nf6 20.Qe7 Nh5 21.Kf1 Nf6 22.Ba4 Nh5 23.Kg1 Nf6 24.Qb7 Nh5 25.Bb3 Nf6 26.Qb5 Re8 27.Qxb4 Rd8 28.Qe7 Rf8 29.Kf1 Nh7 30.Qb4 Re8
+- (6.26) Depth: 47/65 00:04:49 372mN, tb=144909
checked at 3 pv and my old 13 yr old computer hops on Rf6 pretty quick, is that not your answer?

Strange though, at 1 pv it seems to not do so.
User avatar
Ajedrecista
Posts: 1950
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: Nobody's perfect - not even Stockfish.

Post by Ajedrecista »

Hello Jouni and Brian:

Just for the record, it is the position ECM.825 of ECM98.epd according to this source.

I tried SF 15 with 1 thread and Multi-PV = 1 for being deterministic, but with NNUE disabled. It had a hard time but finally succeed at depth 44:

Code: Select all

FEN: 1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - 0 1 

Stockfish_15_x64_popcnt:
Found 145 tablebases
classical evaluation enabled
[...]
 41/93	03:58	 231.883.237	972.065	+3,46	Qe5d4 Kh8g8 Rd6xf6 g7xf6 Qd4xf6 Qb8c7 Qf6xh6 Rd8xd7 Qh6g5+ Kg8f8 Bc6xd7 Qc7xd7 h5h6 Qd7d4 f2f3 Qd4d2+ Kg2h3 Qd2d1 Qg5g7+ Kf8e7 Qg7g2 Qd1d3 Kh3h4 Qd3b1 Qg2g5+ f7f6 Qg5g7+ Ke7e8 Qg7xf6 Qb1h1+ Kh4g5 Qh1g2+ Kg5h5 Qg2xf3+ Kh5g6 Qf3g2+ Qf6g5 Qg2c6+ Kg6h7 Qc6b7+ Kh7h8 Qb7f7 Qg5e5+ Ke8d8 f4f5 Qf7f8+ Kh8h7 Qf8f7+ Qe5g7 Qf7xf5+ Qg7g6 Qf5f4 Qg6e6 a6a5 Kh7g7 Qf4c7+ Qe6f7 Qc7g3+ Qf7g6 Qg3c7+ Kg7g8 Qc7c4+ Qg6f7 Qc4g4+ Kg8h7 Qg4e4+ Kh7g7 Kd8c8 h6h7 Qe4g4+ Qf7g6 Qg4d4+ Kg7f7 Qd4f4+ Qg6f6 Qf4c7+ Kf7g8 Qc7c4+ Kg8h8 Qc4d5 Kh8g7 Qd5d7+ Kg7g8 Qd7d5+ Kg8f8 Qd5c5+ Kf8g7
 42/91-	04:10	 242.599.514	970.153	+3,26	Qe5d4 Kh8g8
 42/91-	04:13	 246.023.335	973.170	+3,06	Qe5d4 Kh8g8
 42/94-	04:19	 254.996.521	982.312	+2,79	Qe5d4 Kh8g8
 42/94+	04:22	 257.707.823	984.000	+3,03	Qe5d4
 42/96	04:32	 270.713.161	993.402	+2,76	Qe5d4 Kh8g8 Rd6xf6 g7xf6 Qd4xf6 Qb8c7 Qf6xh6 Rd8xd7 Qh6g5+ Kg8f8 Bc6xd7 Qc7b7+ f2f3 Qb7xd7 h5h6 Qd7d2+ Kg2h3 Qd2d1 Qg5g7+ Kf8e7 Qg7g2 Qd1d3 Kh3h4 Qd3b1 Qg2g5+ f7f6 Qg5g7+ Ke7e8 Qg7xf6 Qb1h1+ Kh4g4 Qh1g2+ Kg4h5 Qg2xf3+ Kh5g6 Qf3g2+ Qf6g5 Qg2c6+ Kg6g7 Qc6d7+ Kg7h8 Qd7f7 Qg5e5+ Ke8d8 f4f5 Qf7f8+ Kh8h7 Qf8f7+ Qe5g7 Qf7xf5+ Qg7g6 Qf5f4 Qg6e6 a6a5 Kh7g7 Qf4c7+ Qe6f7 Qc7g3+ Qf7g6 Qg3c7+ Kg7g8 Qc7c4+ Qg6f7 Qc4g4+ Kg8h7 Qg4e4+ Kh7g7 Kd8c8 h6h7 Qe4e5+ Qf7f6 Qe5c7+ Kg7h8 Qc7c4 Qf6f5+ Kc8b8 Qf5e5+ Kb8c8 Qe5e8+ Kc8b7 Qe8d7+ Kb7b8 Qd7d6+ Kb8a7 Kh8g7 Qc4g4+ Qd6g6 Qg4d4+ Qg6f6 Qd4g4+ Kg7f8 Qg4c8+ Kf8f7 Qc8c4+ Qf6e6
 43/97-	04:49	 286.829.714	991.618	+2,67	Qe5d4 Kh8g8
 43/97	04:51	 289.416.844	993.385	+2,69	Qe5d4 Kh8g8 Rd6xf6 g7xf6 Qd4xf6 Qb8c7 Qf6xh6 Rd8xd7 Qh6g5+ Kg8f8 Bc6xd7 Qc7b7+ f2f3 Qb7xd7 h5h6 Qd7d2+ Kg2h3 Qd2d1 Qg5g7+ Kf8e7 Qg7g2 Qd1d3 Kh3h4 Qd3b1 Qg2g5+ f7f6 Qg5g7+ Ke7e8 Qg7xf6 Qb1h1+ Kh4g4 Qh1g2+ Kg4h5 Qg2xf3+ Kh5g6 Qf3g2+ Kg6h7 Qg2b7+ Kh7h8 Qb7f7 Qf6e5+ Ke8d8 f4f5 Qf7f8+ Kh8h7 Qf8f7+ Qe5g7 Qf7xf5+ Qg7g6 Qf5f4 Qg6e6 a6a5 Kh7g7 Qf4c7+ Qe6f7 Qc7g3+ Qf7g6 Qg3c7+ Kg7g8 Qc7c4+ Qg6f7 Qc4g4+ Kg8h7 Qg4e4+ Kh7g7 Kd8c8 Qf7f6 Qe4b7+ Kg7g8 Qb7g2+ Kg8h8 Qg2e4 h6h7 Qe4d5 Kh8g7 Qd5d7+ Kg7g8 Qd7d5+ Kg8h8 Qd5c4 Qf6f8+ Kc8b7 Qf8f5 Kb7a8
 44/96-	05:16	 316.173.969	1.000.553	+2,55	Qe5d4 Kh8g8
 44/101-	05:22	 324.480.181	1.007.874	+2,39	Qe5d4 Kh8g8
 44/101+	05:30	 334.551.790	1.013.774	+2,55	Rd6xf6
 44/101+	06:40	 439.295.930	1.098.596	+14,62	Rd6xf6
 44/101	07:05	 484.557.278	1.140.223	+14,75	Rd6xf6 g7xf6 Qe5xf6+ Kh8g8 Qf6xh6 Rd8xd7 Bc6xd7 Qb8d8 Bd7c6 f7f6 Qh6g6+ Kg8h8 Bc6e4 Qd8g8 h5h6 Qg8xg6+ Be4xg6 Kh8g8 Kg2f3 b6b5 a4a5 f6f5 Bg6xf5 Kg8f7 Kf3g4 Kf7e7 Bf5d3 Ke7d6 f4f5 b5b4 Kg4g5 b4b3 h6h7 b3b2 f5f6 Kd6d5
 45/47+	07:39	 548.235.919	1.193.540	+16,59	Rd6xf6
 45/51+	09:14	 721.865.136	1.302.504	+18,88	Rd6xf6
 45/58+	21:55	2.143.668.282	1.629.866	+42,58	Rd6xf6
 45/58	22:10	2.171.062.602	1.631.717	+50,74	Rd6xf6 Kh8g8 Qe5xb8 Rd8xb8 Rf6f5 f7f6 Rf5d5 Rb8d8 Rd5d1 Kg8f7 Rd1e1 b6b5 a4a5 b5b4 Re1e8 Rd8xd7 Bc6xd7 b4b3 Re8b8 Kf7e7 Bd7h3 Ke7d6 Rb8xb3 Kd6c5 Bh3f5 Kc5d6 Rb3b6+ Kd6c5 Rb6xa6 Kc5b5 Bf5d3+ Kb5a4 Ra6a7 g7g6 h5xg6 Ka4b4 a5a6 Kb4c3 Ra7d7
There were 1,618,791 TB hits to the full set of 3, 4 and 5-man Syzygy EGTB.

Looking at the title of the topic... what can be said? Pobody's nerfect but things sometimes get arranged somehow (NNUE disabled in this case).

Image

Image

Source: The Simpsons S06 E16 'Bart vs. Australia' (1995).

Regards from Spain.

Ajedrecista.
CornfedForever
Posts: 597
Joined: Mon Jun 20, 2022 4:08 am
Full name: Brian D. Smith

Re: Nobody's perfect - not even Stockfish

Post by CornfedForever »

The only question to me is - why does SF w/NNUE find the move pretty easily with 3 pv....but not with 1 pv? I am presuming Rf6 is the 'correct' or most expeditious move...
peter
Posts: 3167
Joined: Sat Feb 16, 2008 7:38 am
Full name: Peter Martan

Re: Nobody's perfect - not even Stockfish.

Post by peter »

Ajedrecista wrote: Thu Sep 15, 2022 8:27 pm Just for the record, it is the position ECM.825 of ECM98.epd according to this source.
The game it's taken from, is Olafsson-Unzicker Lugano 1970 according to my database.

In older versions of Hard Talkchess 2020 is once was nr.10, in the remaining 108 of Vincent's latest version nr. 8.

It's a clear best move but problem for engines is, the position is already won with some other moves too:

1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - 0 1

Analysis by Stockfish 110922:

1. +- (152.49): 1.Txf6 gxf6 2.Dxf6+ Kg8 3.Dxh6 Txd7 4.Dg5+ Kf8 5.h6 Dc8 6.Lxd7 Db7+ 7.Kh2 Dxd7 8.Dg7+ Ke7 9.De5+ De6 10.h7 f6 11.Dxe6+ Kxe6 12.h8D Ke7 13.Dg7+ Ke6 14.Db7 b5 15.Dxa6+ Kd7 16.axb5 f5 17.Df6 Kc7 18.Dxf5 Kd8 19.Df7
2. +- (13.45): 1.Kf3 Sg8 2.Kg2 a5 3.Td5 Da7 4.Td3 Sf6 5.Td6 Kg8 6.De7 Dc7 7.Kf3 Kh8 8.De5 b5 9.axb5 Db8 10.Kg2 Dc7 11.Dd4 Kg8 12.b6 Db8 13.b7 a4 14.De5 Txd7 15.Lxd7 Dxb7+ 16.Lc6 Dc7 17.Lxa4 Dc8 18.Lc6 Dg4+ 19.Kf1 Dc8 20.Lf3 Se8 21.Kg2 Sxd6 22.Dxd6 De6 23.Dd8+ Kh7
3. +- (13.24): 1.De7 Sg8 2.De5 a5 3.Td4 Dxe5 4.fxe5 Se7 5.Lb5 Kg8 6.Kf3 g5 7.Ke4 Kf8 8.Td6 Sg8 9.Txb6 Kg7 10.Kd5 Se7+ 11.Kc5 Kf8 12.Txh6 Sg8 13.Td6 Kg7 14.Kc6 Sh6 15.Kc7 Th8 16.Lc6 Tf8 17.Ld5
4. +- (12.16): 1.Kg1 Kh7 2.Kg2 Kg8 3.De7 Dc7 4.Kf3 a5 5.Kg2 Kh8 6.De5 b5 7.axb5 Db8 8.Kf3 Sg8 9.Td3 Dxe5 10.fxe5 Se7 11.Ke4 Kg8 12.Kd4 g5 13.Kc5 Kf8 14.Le4 Sc8 15.Lf5 Se7 16.Lh3 f5 17.b6 Kf7 18.Kd6 Sc8+ 19.Kc6

So winning eval guides the move ordering to the easiest win maybe before the very best one's fully searched through. And with NNUE eval guides search with even more pruning maybe.
Anyhow you're right, NNUE usage unchecked makes SF dev. find the best move at once single primary too.

1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - 0 1

Analysis by Stockfish 110922:

1.Td2 Da7 2.Td3 b5 3.axb5 axb5 4.Lxb5 Da8+ 5.Kg1 Db7 6.De7 Txd7 7.Txd7
Weiß steht deutlich besser: +/- (1.56) Tiefe: 10/16 00:00:00
...
1.Td2 Da7 2.Td3 Kh7 3.Te3 Txd7 4.Df5+ Kg8 5.Lxd7 Db7+ 6.Kg3 Dxd7 7.Dxd7 Sxd7 8.Te8+ Kh7 9.Te7 Sc5
Weiß steht deutlich besser: +- (1.82) Tiefe: 15/22 00:00:00 2356kN
1.Dd4 Dc7 2.Txf6 gxf6 3.Dxf6+ Kg8 4.Dxh6 Txd7 5.Dg5+ Kf8 6.Lxd7 Dxd7 7.h6 Dd4 8.Kg3 b5 9.axb5 axb5 10.Dxb5 Kg8 11.De8+ Kh7 12.Dxf7+ Kxh6 13.f5 Dg7+ 14.Dg6+ Dxg6+ 15.fxg6
Weiß steht deutlich besser: +- (1.79) Tiefe: 16/29 00:00:00 3516kN
...
1.Dd4 Dc7 2.Txf6 gxf6 3.Dxf6+ Kg8 4.Dxh6 Txd7 5.Dg5+ Kf8 6.Lxd7 Dxd7 7.h6 Dd4 8.f3 Dd2+ 9.Kh3 Dd1 10.Dg7+ Ke8 11.Dg8+ Ke7 12.Dg2 Db1 13.Kh4 b5 14.Dg5+ Ke8 15.De5+ Kf8 16.Db8+ Ke7 17.Dc7+ Ke8 18.axb5 axb5 19.Dc6+ Ke7 20.De4+ Dxe4 21.fxe4
Weiß steht deutlich besser: +- (2.12) Tiefe: 24/48 00:00:00 18749kN
1.Txf6 Dxe5 2.fxe5 gxf6 3.exf6 Kh7 4.Kf3 Tb8 5.Ke4 b5 6.Ke5 b4 7.Le4+ Kg8 8.Kd6 b3 9.Kc7 Tf8 10.a5 Kh8 11.d8D
Weiß steht klar auf Gewinn: +- (6.75) Tiefe: 25/56 00:00:00 22240kN
...
1.Txf6 Kg8 2.Tf5 Dxe5 3.Txe5 Kh7 4.Le4+ g6 5.hxg6+ Kg7 6.Te7 Kf6 7.Txf7+ Ke6 8.f5+ Kd6 9.Tf6+ Kc5 10.Tc6+ Kb4 11.Txb6+ Kc4 12.Td6 Kc5 13.Txa6 Txd7 14.f6 Kd4 15.Kf3 h5 16.g7 Tc7 17.Lg6 Tc3+ 18.Kf4 Th3 19.f7 Th2 20.Kf3 h4 21.Tb6 Th3+ 22.Kg4
Weiß steht klar auf Gewinn: +- (31.06 ++) Tiefe: 29/54 00:00:12 833MN

Tried it another two times more, all about as fast as that. 30 threads of 16x3.5GHz, Syzygy Probe Limit 0, 32G hash,
regards
Peter.