berserk13 cannot find the exchange sac of lc0

[Uri Blass]

That's all fine and good, but Berserk 13 does solve this mate in 23 by Peter Krug in a little over 2 minutes on my computer that lots of other engines can't including The Huntsman 1, Stockfish, Crystal, Komodo dragon, (I can't say for lc0 because I don't have it) and a host of others:

[pgn][Event "Mate in 23"]
[Site "Columbus, Ohio"]
[Date "2024.04.09"]
[Round "?"]
[White "Peter Krug"]
[Black "Berserk-13-avx512-pext"]
[Result "1-0"]
[Time "19:28:26"]
[TimeControl "0"]
[SetUp "1"]
[FEN "3K4/2p2p2/2P2P2/4R3/R1p5/1prp2p1/1prp2P1/1kbB4 w - - 0 1"]
[Termination "normal"]
[PlyCount "45"]
[WhiteType "human"]
[BlackType "program"]

1. Rb4 {+M23/74 127 Berserk-13-avx512-pext} Ka2 2. Ra5+ Kb1 3. Rg5 Ka1 4.
Ra4+ Kb1 5. Rg6 fxg6 6. f7 g5 7. f8=B g4 8. Ba3 Ka1 9. Bd6+ Kb1 10. Ra8
cxd6 11. c7 d5 12. c8=N d4 13. Na7 Ka1 14. Nb5+ Kb1 15. Na3+ Ka1 16. Nxc2+
Kb1 17. Nxd4 Rc2 18. Nb5 c3 19. Na3+ Ka2 20. Nxc2+ Kb1 21. Na3+ Ka2 22.
Nb5+ Kb1 23. Nxc3# 1-0
[/pgn]

My point here is that we can ALWAYS find a position that one engine might not be able to solve, but there are others it will solve that will make the other engines look dumb that can't solve it. It has something to do with the pruning methods and you can't account for everything or every possibility without sacrificing something.

Berserk with 6 cores did not have luck on my computer..

FEN: 3K4/2p2p2/2P2P2/4R3/R1p5/1prp2p1/1prp2P1/1kbB4 w - - 0 1

Berserk13:
time -1 start 132401406 alloc 0 max 2147483647 depth 200 timeset 0 searchmoves 0
1/1 00:00 249 249k -11.96 Re5-e8
2/2 00:00 3k 2,688k 0.00 Re5-e8
3/2 00:00 5k 4,863k 0.00 Re5-e8
4/2 00:00 6k 5,824k 0.00 Re5-e8
5/2 00:00 8k 8,034k 0.00 Re5-e8
6/2 00:00 10k 9,760k 0.00 Re5-e8
7/2 00:00 12k 11,854k 0.00 Re5-e8
8/2 00:00 14k 14,123k 0.00 Re5-e8
9/2 00:00 16k 15,575k 0.00 Re5-e8
10/2 00:00 18k 17,986k 0.00 Re5-e8
11/2 00:00 21k 21,154k 0.00 Re5-e8
12/2 00:00 23k 23,330k 0.00 Re5-e8
13/2 00:00 25k 25,060k 0.00 Re5-e8
14/2 00:00 31k 30,602k 0.00 Re5-e8
15/2 00:00 37k 37,429k 0.00 Re5-e8
16/2 00:00 40k 39,742k 0.00 Re5-e8
17/2 00:00 42k 42,109k 0.00 Re5-e8
18/2 00:00 45k 44,764k 0.00 Re5-e8
19/2 00:00 47k 46,682k 0.00 Re5-e8
20/2 00:00 50k 50,150k 0.00 Re5-e8
21/2 00:00 55k 54,537k 0.00 Re5-e8
22/2 00:00 56k 56,064k 0.00 Re5-e8
23/2 00:00 58k 58,211k 0.00 Re5-e8
24/2 00:00 59k 59,267k 0.00 Re5-e8
25/2 00:00 61k 60,574k 0.00 Re5-e8
26/2 00:00 62k 61,866k 0.00 Re5-e8
27/2 00:00 64k 63,647k 0.00 Re5-e8
28/2 00:00 65k 65,338k 0.00 Re5-e8
29/2 00:00 68k 68,013k 0.00 Re5-e8
30/2 00:00 72k 71,950k 0.00 Re5-e8
31/2 00:00 89k 89,143k 0.00 Re5-e8
32/2 00:00 94k 93,896k 0.00 Re5-e8
33/2 00:00 114k 7,612k 0.00 Re5-e8
34/2 00:00 170k 11,324k 0.00 Re5-e8
35/2 00:00 260k 17,357k 0.00 Re5-e8
36/2 00:00 322k 10,373k 0.00 Re5-e8
37/2 00:00 400k 12,918k 0.00 Re5-e8
38/2 00:00 475k 10,100k 0.00 Re5-e8
39/2 00:00 617k 13,119k 0.00 Re5-e8
40/2 00:00 785k 12,664k 0.00 Re5-e8
41/2 00:00 942k 12,071k 0.00 Re5-e8
42/2 00:00 1,123k 11,946k 0.00 Re5-e8
43/2 00:00 1,385k 12,704k 0.00 Re5-e8
44/2 00:00 1,595k 12,763k 0.00 Re5-e8
45/2 00:00 1,703k 13,622k 0.00 Re5-e8
46/2 00:00 1,906k 13,611k 0.00 Re5-e8
47/2 00:00 2,214k 12,870k 0.00 Re5-e8
48/2 00:00 2,799k 13,786k 0.00 Re5-e8
49/2 00:00 3,292k 13,167k 0.00 Re5-e8
50/2 00:00 4,014k 13,514k 0.00 Re5-e8
51/2 00:00 4,645k 13,504k 0.00 Re5-e8
52/2 00:00 5,666k 13,956k 0.00 Re5-e8
53/2 00:00 6,572k 14,012k 0.00 Re5-e8
54/2 00:00 8,060k 14,342k 0.00 Re5-e8
55/2 00:00 9,597k 14,281k 0.00 Re5-e8
56/2 00:00 13,235k 13,659k 0.00 Re5-e8
57/2 00:01 16,903k 13,523k 0.00 Re5-e8
58/2 00:01 21,622k 12,932k 0.00 Re5-e8
59/2 00:01 21,628k 12,936k 0.00 Re5-e8
60/2 00:01 24,260k 12,939k 0.00 Re5-e8
61/2 00:02 29,982k 12,879k 0.00 Re5-e8
62/2 00:02 31,453k 12,822k 0.00 Re5-e8
63/2 00:03 47,306k 12,511k 0.00 Re5-e8
64/2 00:05 65,568k 11,921k 0.00 Re5-e8
65/2 00:05 69,609k 11,693k 0.00 Re5-e8
66/2 00:06 77,335k 11,591k 0.00 Re5-e8
67/2 00:08 100,775k 11,664k 0.00 Re5-e8
68/2 00:08 102,735k 11,637k 0.00 Re5-e8
69/2 00:11 135,506k 11,579k 0.00 Re5-e8
70/2 00:12 139,056k 11,574k 0.00 Re5-e8
71/2 00:14 169,098k 11,404k 0.00 Re5-e8
72/2 00:22 237,018k 10,660k 0.00 Re5-e8
73/2 00:25 264,630k 10,513k 0.00 Re5-e8
74/2 00:33 344,214k 10,314k 0.00 Re5-e8
75/2 00:52 525,277k 10,041k 0.00 Re5-e8
76/2 00:52 525,357k 10,040k 0.00 Re5-e8
77/2 01:00 596,881k 9,873k 0.00 Re5-e8
78/2 01:10 684,605k 9,756k 0.00 Re5-e8
79/2 01:11 698,260k 9,738k 0.00 Re5-e8
80/2 04:31 2,414,750k 8,896k 0.00 Re5-e8
81/2 04:31 2,414,766k 8,896k 0.00 Re5-e8
82/2 04:31 2,414,783k 8,896k 0.00 Re5-e8
83/2 04:31 2,415,454k 8,896k 0.00 Re5-e8
84/2 04:31 2,415,495k 8,896k 0.00 Re5-e8

[Uri Blass]

needed 20 minutes

FEN: 3K4/2p2p2/2P2P2/4R3/R1p5/1prp2p1/1prp2P1/1kbB4 w - - 0 1

Berserk13:
time -1 start 132401406 alloc 0 max 2147483647 depth 200 timeset 0 searchmoves 0
1/1 00:00 249 249k -11.96 Re5-e8
2/2 00:00 3k 2,688k 0.00 Re5-e8
3/2 00:00 5k 4,863k 0.00 Re5-e8
4/2 00:00 6k 5,824k 0.00 Re5-e8
5/2 00:00 8k 8,034k 0.00 Re5-e8
6/2 00:00 10k 9,760k 0.00 Re5-e8
7/2 00:00 12k 11,854k 0.00 Re5-e8
8/2 00:00 14k 14,123k 0.00 Re5-e8
9/2 00:00 16k 15,575k 0.00 Re5-e8
10/2 00:00 18k 17,986k 0.00 Re5-e8
11/2 00:00 21k 21,154k 0.00 Re5-e8
12/2 00:00 23k 23,330k 0.00 Re5-e8
13/2 00:00 25k 25,060k 0.00 Re5-e8
14/2 00:00 31k 30,602k 0.00 Re5-e8
15/2 00:00 37k 37,429k 0.00 Re5-e8
16/2 00:00 40k 39,742k 0.00 Re5-e8
17/2 00:00 42k 42,109k 0.00 Re5-e8
18/2 00:00 45k 44,764k 0.00 Re5-e8
19/2 00:00 47k 46,682k 0.00 Re5-e8
20/2 00:00 50k 50,150k 0.00 Re5-e8
21/2 00:00 55k 54,537k 0.00 Re5-e8
22/2 00:00 56k 56,064k 0.00 Re5-e8
23/2 00:00 58k 58,211k 0.00 Re5-e8
24/2 00:00 59k 59,267k 0.00 Re5-e8
25/2 00:00 61k 60,574k 0.00 Re5-e8
26/2 00:00 62k 61,866k 0.00 Re5-e8
27/2 00:00 64k 63,647k 0.00 Re5-e8
28/2 00:00 65k 65,338k 0.00 Re5-e8
29/2 00:00 68k 68,013k 0.00 Re5-e8
30/2 00:00 72k 71,950k 0.00 Re5-e8
31/2 00:00 89k 89,143k 0.00 Re5-e8
32/2 00:00 94k 93,896k 0.00 Re5-e8
33/2 00:00 114k 7,612k 0.00 Re5-e8
34/2 00:00 170k 11,324k 0.00 Re5-e8
35/2 00:00 260k 17,357k 0.00 Re5-e8
36/2 00:00 322k 10,373k 0.00 Re5-e8
37/2 00:00 400k 12,918k 0.00 Re5-e8
38/2 00:00 475k 10,100k 0.00 Re5-e8
39/2 00:00 617k 13,119k 0.00 Re5-e8
40/2 00:00 785k 12,664k 0.00 Re5-e8
41/2 00:00 942k 12,071k 0.00 Re5-e8
42/2 00:00 1,123k 11,946k 0.00 Re5-e8
43/2 00:00 1,385k 12,704k 0.00 Re5-e8
44/2 00:00 1,595k 12,763k 0.00 Re5-e8
45/2 00:00 1,703k 13,622k 0.00 Re5-e8
46/2 00:00 1,906k 13,611k 0.00 Re5-e8
47/2 00:00 2,214k 12,870k 0.00 Re5-e8
48/2 00:00 2,799k 13,786k 0.00 Re5-e8
49/2 00:00 3,292k 13,167k 0.00 Re5-e8
50/2 00:00 4,014k 13,514k 0.00 Re5-e8
51/2 00:00 4,645k 13,504k 0.00 Re5-e8
52/2 00:00 5,666k 13,956k 0.00 Re5-e8
53/2 00:00 6,572k 14,012k 0.00 Re5-e8
54/2 00:00 8,060k 14,342k 0.00 Re5-e8
55/2 00:00 9,597k 14,281k 0.00 Re5-e8
56/2 00:00 13,235k 13,659k 0.00 Re5-e8
57/2 00:01 16,903k 13,523k 0.00 Re5-e8
58/2 00:01 21,622k 12,932k 0.00 Re5-e8
59/2 00:01 21,628k 12,936k 0.00 Re5-e8
60/2 00:01 24,260k 12,939k 0.00 Re5-e8
61/2 00:02 29,982k 12,879k 0.00 Re5-e8
62/2 00:02 31,453k 12,822k 0.00 Re5-e8
63/2 00:03 47,306k 12,511k 0.00 Re5-e8
64/2 00:05 65,568k 11,921k 0.00 Re5-e8
65/2 00:05 69,609k 11,693k 0.00 Re5-e8
66/2 00:06 77,335k 11,591k 0.00 Re5-e8
67/2 00:08 100,775k 11,664k 0.00 Re5-e8
68/2 00:08 102,735k 11,637k 0.00 Re5-e8
69/2 00:11 135,506k 11,579k 0.00 Re5-e8
70/2 00:12 139,056k 11,574k 0.00 Re5-e8
71/2 00:14 169,098k 11,404k 0.00 Re5-e8
72/2 00:22 237,018k 10,660k 0.00 Re5-e8
73/2 00:25 264,630k 10,513k 0.00 Re5-e8
74/2 00:33 344,214k 10,314k 0.00 Re5-e8
75/2 00:52 525,277k 10,041k 0.00 Re5-e8
76/2 00:52 525,357k 10,040k 0.00 Re5-e8
77/2 01:00 596,881k 9,873k 0.00 Re5-e8
78/2 01:10 684,605k 9,756k 0.00 Re5-e8
79/2 01:11 698,260k 9,738k 0.00 Re5-e8
80/2 04:31 2,414,750k 8,896k 0.00 Re5-e8
81/2 04:31 2,414,766k 8,896k 0.00 Re5-e8
82/2 04:31 2,414,783k 8,896k 0.00 Re5-e8
83/2 04:31 2,415,454k 8,896k 0.00 Re5-e8
84/2 04:31 2,415,495k 8,896k 0.00 Re5-e8
85/2 17:23 8,673,481k 8,309k 0.00 Re5-e8
86/2 17:26 8,701,447k 8,312k 0.00 Re5-e8
87/2 18:48 9,407,879k 8,335k 0.00 Re5-e8
88/46+ 18:53 9,453,794k 8,339k +0.05 Ra4-b4
88/46+ 18:57 9,489,388k 8,344k +0.10 Ra4-b4
88/46+ 20:01 10,262,536k 8,540k +11.17 Ra4-b4
88/46 20:40 10,611,748k 8,554k +M23 Ra4-b4 Kb1-a2 Re5-a5+ Ka2-b1 Ra5-g5 Kb1-a2 Rb4-a4+ Ka2-b1 Rg5-g6 f7xg6 f6-f7 g6-g5 f7-f8B g5-g4 Bf8-a3 Kb1-a2 Ba3-d6+ Ka2-b1 Ra4-a8 c7xd6 c6-c7 d6-d5 c7-c8N d5-d4 Nc8-a7 Kb1-a1 Na7-b5+ Ka1-b1 Nb5-a3+ Kb1-a1 Na3xc2+ Ka1-b1 Nc2xd4 Rc3-c2 Nd4-b5 c4-c3 Nb5-a3+ Kb1-a2 Na3xc2+ Ka2-b1 Nc2-a3+ Kb1-a2 Na3-b5+ Ka2-b1 Nb5xc3+

[Uri Blass]

I can add that I do not think that berserk with the 0.00 evaluation can be proud about the mate because it is easy to understand that white has at least a draw and maybe more than it even without seeing the mate so a good engine should give more than 0.00 for white and prefer a move that force a draw later and not immediate stalemate.

[acase]

needed 20 minutes

FEN: 3K4/2p2p2/2P2P2/4R3/R1p5/1prp2p1/1prp2P1/1kbB4 w - - 0 1

Berserk13:
time -1 start 132401406 alloc 0 max 2147483647 depth 200 timeset 0 searchmoves 0
1/1 00:00 249 249k -11.96 Re5-e8
2/2 00:00 3k 2,688k 0.00 Re5-e8
3/2 00:00 5k 4,863k 0.00 Re5-e8
4/2 00:00 6k 5,824k 0.00 Re5-e8
5/2 00:00 8k 8,034k 0.00 Re5-e8
6/2 00:00 10k 9,760k 0.00 Re5-e8
7/2 00:00 12k 11,854k 0.00 Re5-e8
8/2 00:00 14k 14,123k 0.00 Re5-e8
9/2 00:00 16k 15,575k 0.00 Re5-e8
10/2 00:00 18k 17,986k 0.00 Re5-e8
11/2 00:00 21k 21,154k 0.00 Re5-e8
12/2 00:00 23k 23,330k 0.00 Re5-e8
13/2 00:00 25k 25,060k 0.00 Re5-e8
14/2 00:00 31k 30,602k 0.00 Re5-e8
15/2 00:00 37k 37,429k 0.00 Re5-e8
16/2 00:00 40k 39,742k 0.00 Re5-e8
17/2 00:00 42k 42,109k 0.00 Re5-e8
18/2 00:00 45k 44,764k 0.00 Re5-e8
19/2 00:00 47k 46,682k 0.00 Re5-e8
20/2 00:00 50k 50,150k 0.00 Re5-e8
21/2 00:00 55k 54,537k 0.00 Re5-e8
22/2 00:00 56k 56,064k 0.00 Re5-e8
23/2 00:00 58k 58,211k 0.00 Re5-e8
24/2 00:00 59k 59,267k 0.00 Re5-e8
25/2 00:00 61k 60,574k 0.00 Re5-e8
26/2 00:00 62k 61,866k 0.00 Re5-e8
27/2 00:00 64k 63,647k 0.00 Re5-e8
28/2 00:00 65k 65,338k 0.00 Re5-e8
29/2 00:00 68k 68,013k 0.00 Re5-e8
30/2 00:00 72k 71,950k 0.00 Re5-e8
31/2 00:00 89k 89,143k 0.00 Re5-e8
32/2 00:00 94k 93,896k 0.00 Re5-e8
33/2 00:00 114k 7,612k 0.00 Re5-e8
34/2 00:00 170k 11,324k 0.00 Re5-e8
35/2 00:00 260k 17,357k 0.00 Re5-e8
36/2 00:00 322k 10,373k 0.00 Re5-e8
37/2 00:00 400k 12,918k 0.00 Re5-e8
38/2 00:00 475k 10,100k 0.00 Re5-e8
39/2 00:00 617k 13,119k 0.00 Re5-e8
40/2 00:00 785k 12,664k 0.00 Re5-e8
41/2 00:00 942k 12,071k 0.00 Re5-e8
42/2 00:00 1,123k 11,946k 0.00 Re5-e8
43/2 00:00 1,385k 12,704k 0.00 Re5-e8
44/2 00:00 1,595k 12,763k 0.00 Re5-e8
45/2 00:00 1,703k 13,622k 0.00 Re5-e8
46/2 00:00 1,906k 13,611k 0.00 Re5-e8
47/2 00:00 2,214k 12,870k 0.00 Re5-e8
48/2 00:00 2,799k 13,786k 0.00 Re5-e8
49/2 00:00 3,292k 13,167k 0.00 Re5-e8
50/2 00:00 4,014k 13,514k 0.00 Re5-e8
51/2 00:00 4,645k 13,504k 0.00 Re5-e8
52/2 00:00 5,666k 13,956k 0.00 Re5-e8
53/2 00:00 6,572k 14,012k 0.00 Re5-e8
54/2 00:00 8,060k 14,342k 0.00 Re5-e8
55/2 00:00 9,597k 14,281k 0.00 Re5-e8
56/2 00:00 13,235k 13,659k 0.00 Re5-e8
57/2 00:01 16,903k 13,523k 0.00 Re5-e8
58/2 00:01 21,622k 12,932k 0.00 Re5-e8
59/2 00:01 21,628k 12,936k 0.00 Re5-e8
60/2 00:01 24,260k 12,939k 0.00 Re5-e8
61/2 00:02 29,982k 12,879k 0.00 Re5-e8
62/2 00:02 31,453k 12,822k 0.00 Re5-e8
63/2 00:03 47,306k 12,511k 0.00 Re5-e8
64/2 00:05 65,568k 11,921k 0.00 Re5-e8
65/2 00:05 69,609k 11,693k 0.00 Re5-e8
66/2 00:06 77,335k 11,591k 0.00 Re5-e8
67/2 00:08 100,775k 11,664k 0.00 Re5-e8
68/2 00:08 102,735k 11,637k 0.00 Re5-e8
69/2 00:11 135,506k 11,579k 0.00 Re5-e8
70/2 00:12 139,056k 11,574k 0.00 Re5-e8
71/2 00:14 169,098k 11,404k 0.00 Re5-e8
72/2 00:22 237,018k 10,660k 0.00 Re5-e8
73/2 00:25 264,630k 10,513k 0.00 Re5-e8
74/2 00:33 344,214k 10,314k 0.00 Re5-e8
75/2 00:52 525,277k 10,041k 0.00 Re5-e8
76/2 00:52 525,357k 10,040k 0.00 Re5-e8
77/2 01:00 596,881k 9,873k 0.00 Re5-e8
78/2 01:10 684,605k 9,756k 0.00 Re5-e8
79/2 01:11 698,260k 9,738k 0.00 Re5-e8
80/2 04:31 2,414,750k 8,896k 0.00 Re5-e8
81/2 04:31 2,414,766k 8,896k 0.00 Re5-e8
82/2 04:31 2,414,783k 8,896k 0.00 Re5-e8
83/2 04:31 2,415,454k 8,896k 0.00 Re5-e8
84/2 04:31 2,415,495k 8,896k 0.00 Re5-e8
85/2 17:23 8,673,481k 8,309k 0.00 Re5-e8
86/2 17:26 8,701,447k 8,312k 0.00 Re5-e8
87/2 18:48 9,407,879k 8,335k 0.00 Re5-e8
88/46+ 18:53 9,453,794k 8,339k +0.05 Ra4-b4
88/46+ 18:57 9,489,388k 8,344k +0.10 Ra4-b4
88/46+ 20:01 10,262,536k 8,540k +11.17 Ra4-b4
88/46 20:40 10,611,748k 8,554k +M23 Ra4-b4 Kb1-a2 Re5-a5+ Ka2-b1 Ra5-g5 Kb1-a2 Rb4-a4+ Ka2-b1 Rg5-g6 f7xg6 f6-f7 g6-g5 f7-f8B g5-g4 Bf8-a3 Kb1-a2 Ba3-d6+ Ka2-b1 Ra4-a8 c7xd6 c6-c7 d6-d5 c7-c8N d5-d4 Nc8-a7 Kb1-a1 Na7-b5+ Ka1-b1 Nb5-a3+ Kb1-a1 Na3xc2+ Ka1-b1 Nc2xd4 Rc3-c2 Nd4-b5 c4-c3 Nb5-a3+ Kb1-a2 Na3xc2+ Ka2-b1 Nc2-a3+ Kb1-a2 Na3-b5+ Ka2-b1 Nb5xc3+

Not bad for a 6 core computer, I was using 32 cores and 64 threads.

[Jouni]

Analysis by The Huntsman 1 and 6 cores:
...
1.Rb4 Ka2 2.Ra5+ Kb1 3.Rg5 Ka2 4.Ra4+ Kb1 5.Rg6 fxg6 6.f7 g5 7.f8B g4 8.Ba3 Ka2 9.Bd6+ Kb1 10.Ra8 cxd6 11.c7 d5 12.c8N d4 13.Na7 Ka2 14.Nb5+ Kb1 15.Na3+ Ka1 16.Nxc2+ Kb1 17.Nxd4 Rc2 18.Nb5 c3 19.Na3+ Ka1 20.Nxc2+ Kb1 21.Na3+ Ka1 22.Nb5+ Kb1 23.Nxc3#
+- (#23) Depth: 52/46 00:00:21 189mN

[acase]

Analysis by The Huntsman 1 and 6 cores:
...
1.Rb4 Ka2 2.Ra5+ Kb1 3.Rg5 Ka2 4.Ra4+ Kb1 5.Rg6 fxg6 6.f7 g5 7.f8B g4 8.Ba3 Ka2 9.Bd6+ Kb1 10.Ra8 cxd6 11.c7 d5 12.c8N d4 13.Na7 Ka2 14.Nb5+ Kb1 15.Na3+ Ka1 16.Nxc2+ Kb1 17.Nxd4 Rc2 18.Nb5 c3 19.Na3+ Ka1 20.Nxc2+ Kb1 21.Na3+ Ka1 22.Nb5+ Kb1 23.Nxc3#
+- (#23) Depth: 52/46 00:00:21 189mN

That's news to me, when I tried The Huntsman 1 several times it could only find a mate in 37, and certainly not in 21 seconds. The only thing I can think of is that I may have done different is that I have classical evaluation rather than NNUE evaluation. I'll have to check it when I get home.

[peter]

Analysis by The Huntsman 1 and 6 cores:
...
1.Rb4 Ka2 2.Ra5+ Kb1 3.Rg5 Ka2 4.Ra4+ Kb1 5.Rg6 fxg6 6.f7 g5 7.f8B g4 8.Ba3 Ka2 9.Bd6+ Kb1 10.Ra8 cxd6 11.c7 d5 12.c8N d4 13.Na7 Ka2 14.Nb5+ Kb1 15.Na3+ Ka1 16.Nxc2+ Kb1 17.Nxd4 Rc2 18.Nb5 c3 19.Na3+ Ka1 20.Nxc2+ Kb1 21.Na3+ Ka1 22.Nb5+ Kb1 23.Nxc3#
+- (#23) Depth: 52/46 00:00:21 189mN

That's news to me, when I tried The Huntsman 1 several times it could only find a mate in 37, and certainly not in 21 seconds. The only thing I can think of is that I may have done different is that I have classical evaluation rather than NNUE evaluation. I'll have to check it when I get home.

NNUE is switched off by checking MateSearch- option, even if NNUE is checked too.
Maybe it was a lucky run of Jouni's Huntsman or he didn't copy from time count after start with empty hash. At a single trial of mine with 30 threads Huntsman too starts with #38 and takes some time or Forward- Backward to see the detours, regards

[acase]

Analysis by The Huntsman 1 and 6 cores:
...
1.Rb4 Ka2 2.Ra5+ Kb1 3.Rg5 Ka2 4.Ra4+ Kb1 5.Rg6 fxg6 6.f7 g5 7.f8B g4 8.Ba3 Ka2 9.Bd6+ Kb1 10.Ra8 cxd6 11.c7 d5 12.c8N d4 13.Na7 Ka2 14.Nb5+ Kb1 15.Na3+ Ka1 16.Nxc2+ Kb1 17.Nxd4 Rc2 18.Nb5 c3 19.Na3+ Ka1 20.Nxc2+ Kb1 21.Na3+ Ka1 22.Nb5+ Kb1 23.Nxc3#
+- (#23) Depth: 52/46 00:00:21 189mN

That's news to me, when I tried The Huntsman 1 several times it could only find a mate in 37, and certainly not in 21 seconds. The only thing I can think of is that I may have done different is that I have classical evaluation rather than NNUE evaluation. I'll have to check it when I get home.

NNUE is switched off by checking MateSearch- option, even if NNUE is checked too.
Maybe it was a lucky run of Jouni's Huntsman or he didn't copy from time count after start with empty hash. At a single trial of mine with 30 threads Huntsman too starts with #38 and takes some time or Forward- Backward to see the detours, regards

Thanks Peter, you're right. I just had a lucky 7 second run, and when I tried again after clearing the hash and restarting The Huntsman 1 it's showing a mate in 36 after 4+ minutes.

FEN: 3K4/2p2p2/2P2P2/4R3/R1p5/1prp2p1/1prp2P1/1kbB4 w - - 0 1

The Huntsman 1 bmi2:
classical evaluation enabled

109/72 04:33 13,038,495k 47,656k +M36 1.Ra4-b4 Kb1-a2 2.Re5-a5+ Ka2-b1 3.Ra5-d5 Kb1-a2 4.Rb4-a4+ Ka2-b1 5.Rd5-d6 c7xd6 6.c6-c7 d6-d5 7.c7-c8Q d5-d4 8.Ra4-b4 Kb1-a1 9.Qc8-a6+ Ka1-b1 10.Qa6-b6 Kb1-a2 11.Qb6-a7+ Ka2-b1 12.Qa7xf7 Kb1-a2 13.Qf7-a7+ Ka2-b1 14.Qa7xd4 Kb1-a2 15.Rb4-a4+ Ka2-b1 16.Qd4xc3 Rc2xc3 17.f6-f7 Rc3-c2 18.f7-f8Q c4-c3 19.Ra4-g4 Kb1-a2 20.Qf8-d6 b2-b1N 21.Qd6xd3 Nb1-a3 22.Rg4-b4 Ka2-a1 23.Rb4xb3 Rc2-a2 24.Qd3xc3+ Bc1-b2 25.Qc3-d3 Na3-b1 26.Rb3-b8 Nb1-a3 27.Kd8-e8 Bb2-e5 28.Rb8-b7 Be5-b2 29.Bd1-b3 Bb2-c1 30.Bb3xa2 Ka1xa2 31.Qd3-b3+ Ka2-a1 32.Ke8-d7 d2-d1Q+ 33.Qb3xd1 Ka1-a2 34.Qd1-b3+ Ka2-a1 35.Rb7-b4 Na3-b5 36.Qb3-b1+

[peter]

BTW, just in case you don't know it, the study is in Chest.epd too with these parameters

Parameter in Datenbank gefunden: #23; 00:01 @ C2/R0/K3/P2/X2;

With these the solution- line is given at once.

[towforce]

I think the issue is a bit deeper than that. There apparently is a trade-off between average performance and reliability. And engine developers tend to only focus on Elo, which is an average of the move quality over many thousands of games and thus many millions of positions. If they can make the engine find the best move in 99.9% of these positions 1% faster, they don't care if that makes it a factor 5 slower for the remaining 0.1% of positions. Because on average it would still be marginally faster. But from the perspective of reliability this is of course a large degradation of the engine quality.

"A man waded through a river that was 1m deep, and got across safely. A statician waded through a river that was on average 0.5m deep. He drowned."

What makes it even worse is that the penalty for having difficulty with some of the positions does saturate. Once the engine needs more time to find the decisive move than what was available in the test game, it simply loses that game. It doesn't matter if it would have taken it 10 times as much time as was available to find the move, or 1000 times as much, or even if it would have not been able to find it at all, no matter how long it could think. The developers are more than willing to sacrifice the engine's ability to solve positions that rarely occur in games for marginal Elo inprovement, and the saturation effect makes it quite easy to do that.

I don't think it is true that there are always positions where an engine must perform very poorly. It was a choice of the developers to completely sacrifice such positions for the 'greater good' of marginal Elo improvement.

So these top engines are not really good engines, in the same sense as that a river with an average depth of 0.2m is not a ford, and that one with an average depth of 0.19m would be a highly superior ford. They have become low-quality products that you cannot rely on. While for analysis it usually isn't very important whether you get the correct result 10% faster on average; it is much more important to always get it in reasonable time.

Suppose you had to make a 100km car trip, and had the choice between two routes. One (the interstate highway) would allways bring you to your destination in 60 min. The shortcut would bring you there in 50 min, by taking a private ferry near its end. But once every 14 days the skipper takes an unscheduled day off, the ferry doesn't run, and you would have to turn back to take the other route anyway, which in total takes you 42 + 42 + 60 = 144min. So 84 min extra with a 1-in-14 probability is on average 6min delay, so the average travel time for taking the ferry would be 56min, 4 min faster than the highway. Which route would you prefer?

Very interesting and thought provoking post. It keeps popping into my head, and just now, it came with a new thought: this MUST be an exploitable weakness! Guide these auto-tuned engines into the types of positions where they're likely to miss something important due to the excessive selectivity of their search, and, in the time available, they'll be blinded to important aspects of the position that your slower, but more thorough, engine will see.

I'm thinking: complex positions (this is my plan to blind most chess engines because it dramatically reduces search depth by increasing the size of the search tree - but engines which are excessively selective in their search would be especially vulnerable).