Mate Test Suite

[jhellis3]

The first position is solved by f4xe5 too.

No, f4xe5 does not result in a mate in 5.

[Henk]

Only after disabling null move Skipper is able to solve the first position. Yet another argument to remove null move from Skipper. Not much of the search will be left over that can be completely trusted: PVS /alpha beta, TT perhaps. The only allowed bug is a too shallow search. But better one bug than sixty.

[zullil]

The first position is solved by f4xe5 too. Nearly any engine will win this position. Same holds for other positions. So no need for elo loss.

And Stockfish resolves it to mate (with 16 threads):

Code: Select all

info depth 50 seldepth 55 multipv 1 score mate 20 nodes 48430417869 nps 29490929 hashfull 999 tbhits 0 time 1642214 pv f4e5 d7c6 e5e6 f5f4 g2f4 g4f4 e6e7 c6d7 e7e8q d7e8 c7e8 f4e5 a3a4 e5f4 e8d6 f4g4 e3e4 g4f3 b4b5 a6b5 a4b5 f3g3 d6c4 g3f3 e4e6 f3f4 e6f6 f4g3 b2a3 g3h4 f6f5 h4g4 c4e3 g4g3 a3d6 g3h4 b1b2 h4h3 f5h5

But try this one instead.
[d]8/2Nbr3/pp6/5p1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - - 0 1
Can Stockfish find a win now?

[Henk]

By the way even Skipper solves this mate in 5 in 27 seconds.

[Henk]

Second position costs Skipper 315 seconds. So its not quick as expected.

[d] 8/8/2B1N3/3rp3/4k2K/7Q/2r3Pn/1b1N4 w - -

[Henk]

It can't solve third position because of 'out of memory' bug. Probably because of TT. TT stored too much. I don't know yet.

[BBauer]

I tried it with a modification:

Code: Select all

	Depth R;
        if (depth < 12) R = (2 + depth/2 ) * ONE_PLY;  
	else                 R = (8 + depth/12) * ONE_PLY;

and got:

Code: Select all

FEN: 8/2Nbr3/pp6/5p1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - - 0 1

St-mod-15-05-08:
 8/10	00:00	 10k	866k	+3,92	Sc7-d5 Te7-e6 Te3-e1 Te6-h6 Sg2-e3+ Kg4xh4 Se3xc4 b6-b5 Sc4-e3
 9/12	00:00	 21k	1.099k	+4,08	Sc7-d5 Te7-f7 Sd5xb6 Ld7-e6 Te3-e1 Kg4-g3 Sg2-e3 Kg3xf4 Sb6xc4
 10/12	00:00	 36k	1.285k	+4,62	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 Td6-e6 Se7xf5 Kf3xf4
 11/16	00:00	 63k	1.360k	+4,63	Sc7-d5 Te7-f7 Sd5xb6 Ld7-e6 Te3-e1 Kg4-g3 Sg2-e3 Kg3xf4 Sb6xc4 Kf4-g3 Te1-h1
 12/16	00:00	 107k	1.461k	+4,61	Sc7-d5 Te7-e6 Te3-e1 Te6-d6 Sg2-e3+ Kg4xh4 Te1-h1+ Kh4-g3 Th1xh5 Kg3-f3 Sd5-e7 Td6-f6 Th5-h3+ Kf3xf4 Se7-d5+ Kf4-e5
 13/24	00:00	 308k	1.517k	+4,38	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 b6-b5 Se3xf5 Ld7xf5 Se7xf5 Td6-d7 Sf5-d4+ Kf3xf4 Sd4-e6+ Kf4-f5 Se6-c5 Td7-e7
 14/24	00:00	 359k	1.496k	+4,38	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 b6-b5 Te1-g1 Td6-f6 Tg1-g5 Kf3-e2 Lb2-c1 Ke2-f3 Se7xf5 Ld7xf5
 15/24	00:00	 460k	1.534k	+4,49	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 b6-b5 Te1-g1 Td6-f6 Tg1-g5 Ld7-e6 Se7xf5 Le6xf5 Se3xf5
 16/28	00:00	 1.254k	1.921k	+4,56	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 b6-b5 Se3xf5 Ld7xf5 Se7xf5 Td6-d7 Sf5-e3 Kf3xf4 Te1-f1+ Kf4-g3 a3-a4 b5xa4 Se3xc4
 17/28	00:00	 1.680k	1.979k	+4,51	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Sd5-e7 b6-b5 Se7xf5 Ld7xf5 Se3xf5 Td6-d7 Sf5-e3 Kf3xf4 Te1-f1+ Kf4-g3 a3-a4 b5xa4 Tf1-f6
 18/28	00:02	 4.462k	2.004k	+4,82	Sc7-d5 Te7-e6 Te3-e1 Te6-d6 Sd5-e3+ Kg4-g3 Se3xc4 Td6-h6 Sg2-e3 Kg3xh4 Sc4-e5 Ld7-e6 Se5-c6 Kh4-g3 Te1-g1+ Kg3-f2 Tg1-h1 Kf2-g3 Sc6-d4 Kg3xf4 Th1-f1+ Kf4-e5
 19/28	00:02	 5.655k	2.057k	+4,68	Sc7-d5 Te7-e6 Te3-e1 Te6-h6 Te1-g1 b6-b5 Sg2-e3+ Kg4-f3 Tg1-g5 Ld7-e6 Sd5-e7 Kf3xf4 Se7xf5 Le6xf5 Se3xf5 Th6-h8 Sf5-e3 Kf4-f3 Kb1-c1 Kf3-e2 Tg5-g6 Th8-h7
 20/28+	00:03	 6.359k	2.067k	+4,81	Sc7-d5
 20/28+	00:03	 6.791k	2.053k	+4,90	Sc7-d5
 20/29-	00:04	 8.342k	2.082k	+4,82	Sc7-d5 Te7-e6
 20/29+	00:04	 8.577k	2.068k	+4,93	Sc7-d5
 20/29	00:04	 9.313k	2.069k	+4,88	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Te1-g1 b6-b5 Tg1-g5 Td6xd5 Se3xd5 Ld7-e6 Sd5-e3 Kf3xf4 Tg5xh5 Kf4-e5 Kb1-c1 Ke5-f6 Th5-g5 Le6-d7 Kc1-b1 Ld7-c8 Se3-g2 Lc8-e6
 21/29+	00:04	 9.877k	2.064k	+4,94	Sc7-d5
 21/29	00:05	 11.241k	2.075k	+4,95	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Te1-g1 b6-b5 Tg1-g5 Td6xd5 Se3xd5 Ld7-e6 Sd5-e3 Kf3xf4 Tg5xh5 Kf4-e5 Kb1-c1 Ke5-f6 Th5-g5 Le6-d7 Se3-d5+ Kf6-e5 Sd5-c7 Ke5-f4 Tg5-g7
 22/29+	00:06	 13.972k	2.089k	+5,01	Sc7-d5
 22/30	00:09	 20.030k	2.122k	+5,06	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Te1-g1 b6-b5 Tg1-g5 Td6xd5 Se3xd5 Ld7-e6 Sd5-e3 Kf3xf4 Tg5xh5 Kf4-e5 Kb1-c1 Ke5-f6 Th5-g5 Le6-d7 a3-a4 b5xa4 Se3xc4 f5-f4 Tg5-e5 a4-a3 Lb2xa3
 23/32+	00:11	 24.767k	2.144k	+5,13	Sc7-d5
 23/32	00:13	 29.331k	2.150k	+5,08	Sc7-d5 Te7-e6 Te3-e1 Kg4-f3 Sg2-e3 Te6-d6 Te1-g1 b6-b5 Tg1-g5 Td6xd5 Se3xd5 Ld7-e6 Sd5-e3 Kf3xf4 Tg5xh5 Kf4-e5 Kb1-c1 Ke5-f6 Th5-g5 Le6-c8 Se3-d5+ Kf6-e5 Sd5-c7 Ke5-f6 Tg5-g8 Lc8-d7 Sc7xa6 f5-f4 Sa6-c5
 24/35+	00:17	 37.318k	2.176k	+5,14	Sc7-d5
 24/35+	00:20	 44.530k	2.177k	+5,20	b4-b5
 24/35+	00:29	 63.999k	2.207k	+8,18	b4-b5
 24/35	00:34	 76.398k	2.233k	+8,77	b4-b5 Te7-f7 b5xa6 b6-b5 a6-a7 Ld7-c6 a7-a8D Lc6xa8 Sc7xa8 Tf7-b7 Kb1-c1 b5-b4 c3xb4 c4-c3 Lb2xc3 Tb7xb4 Sa8-c7 b3-b2+ Kc1-b1 Tb4-b5 Te3-e1 Tb5-c5 Sc7-e6 Tc5xc3 Sg2-e3+ Kg4-f3 d2xc3
 25/35+	00:34	 76.654k	2.233k	+8,84	b4-b5
 25/35+	00:34	 76.798k	2.233k	+8,90	b4-b5
 25/35+	00:57	 126.327k	2.181k	+13,45	b4-b5
 25/35	01:05	 144.116k	2.191k	+13,45	b4-b5 Te7-e5 f4xe5 a6xb5 Sc7-d5 f5-f4 Sg2xf4 Kg4-f5 e5-e6 Ld7xe6 Sf4xe6 Kf5xe6 Sd5-c7+ Ke6-e5 Sc7xb5 Ke5-d5 a3-a4 Kd5-e5 Lb2-a3 Ke5-d5 Sb5-d6 b3-b2 Kb1xb2 Kd5-c6 Te3xe4 Kc6-c7 Te4-d4
 26/35+	01:09	 152.278k	2.196k	+13,51	b4-b5
 26/35+	01:10	 154.895k	2.194k	+13,58	b4-b5
 26/35+	01:15	 162.225k	2.144k	+14,02	b4-b5
 26/35	01:47	 230.709k	2.153k	+14,08	b4-b5 Te7-e5 f4xe5 a6xb5 Sc7-d5 f5-f4 Sg2xf4 Kg4-f5 e5-e6 Ld7xe6 Sf4xe6 Kf5xe6 Sd5-c7+ Ke6-f5 Sc7xb5 Kf5-g4 Te3xe4+ Kg4-f3 Te4xc4 Kf3-g3 Tc4-b4 Kg3-f2 a3-a4 Kf2-e2 Kb1-c1 Ke2-f2 c3-c4 Kf2-g3 Sb5-d6 Kg3-h3 Tb4xb6 Kh3xh4
 27/36+	01:52	 244.071k	2.160k	+14,14	b4-b5
 27/36+	01:54	 247.109k	2.163k	+14,20	b4-b5
 27/37+	03:56	 538.259k	2.271k	+94,60	b4-b5
 27/37	04:00	 548.062k	2.276k	+M5	Kb1-c1 Ld7-c6 Kc1-d1 Te7xc7 Kd1-e1 Lc6-b5 Ke1-f2 Tc7-a7 Te3-g3+
 28/37	04:00	 548.111k	2.276k	+M5	Kb1-c1 Ld7-c6 Kc1-d1 Te7xc7 Kd1-e1 Lc6-b5 Ke1-f2 Tc7-a7 Te3-g3+
 29/37	04:00	 548.144k	2.276k	+M5	Kb1-c1 Ld7-c6 Kc1-d1 Te7xc7 Kd1-e1 Lc6-b5 Ke1-f2 Tc7-a7 Te3-g3+
 30/37	04:00	 548.182k	2.276k	+M5	Kb1-c1 Ld7-c6 Kc1-d1 Te7xc7 Kd1-e1 Lc6-b5 Ke1-f2 Tc7-a7 Te3-g3+
 

kind regards
bernhard

[zullil]

TLDR: A program primarily designed to solve mates and not play chess can solve mates, news at 11 ...

I would have thought this all to be rather self-evident, but I digress...

A engine designed to win chess games and not strive for minimal mates struggles to find minimal mates. Film at 11.

I think I understand the point of your post, but some of your positions seem silly. For instance, the second one, which amounts to "How long for your engine to see mate-in-4 here instead of mate-in-7?"
[d]8/8/2B1N3/3rp2K/4k3/7Q/b1r3Pn/3N4 w - - 2 2

Here's the answer for the latest Stockfish, with one thread and 512 MB hash:

Code: Select all

info depth 53 seldepth 24 multipv 1 score mate 4 nodes 132265975 nps 3147840 hashfull 727 tbhits 0 time 42018 pv h5h6 a2b3 h6h7 b3c4 h7h8 c4e2 d1f2

But it finds a (non-minimal) forced mate in a fraction of a second.

[kgburcham]

Louis amazing test position because it is so few moves.

[d] 8/2Nbr3/pp6/5p1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - -

8 processor(s) found, POPCNT available
NUMA configuration with 1 node(s), offset 0
8192 MB Hash
Engine: Houdini 4 Pro x64 (8192 MB)
by Robert Houdart
19/49 0:00 +4.70 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rc6 4.Bc1 Be6 5.Ne7 Rc7 6.N7xf5 Bxf5 7.Nxf5 Kxf4 8.Nd6 e3 9.Rxe3 b5 10.Re4+ Kg3 11.Kb2 Kf3 12.Re5 Kg4 13.Rc5 Re7 14.Nf5 (19.402.039) 19818
20/53 0:01 +4.86 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rc6 4.Bc1 Be6 5.Ne7 Rc7 6.N7xf5 Bxf5 7.Nxf5 b5 8.Nd4+ Kg4 9.Rxe4 Kxh4 10.Re5 Rf7 11.Nf5+ Kh3 12.Nh6 Rxf4 13.Rxh5+ Rh4 14.Rxh4+ (22.367.780) 19812
21/66 0:01 +4.89 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rd6 4.Bc1 b5 5.Rf1+ Kg3 6.Rg1+ Kf2 7.Rg5 Rh6 8.Kb2 Be6 9.Nxf5 Bxf5 10.Rxf5 Kg3 11.Re5 Kxh4 12.f5 Kg5 13.f6+ Kg6 14.Rxe4 (28.307.894) 19837
22/66 0:01 +4.87 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rd6 4.Bc1 b5 5.Rf1+ Kg3 6.Rg1+ Kf2 7.Rg5 Rh6 8.Kb2 Be6 9.Nxf5 Bxf5 10.Rxf5 Kg3 11.Re5 Kxh4 12.f5 Kg5 13.f6+ Kg6 14.Rxe4 (32.580.348) 20037
23/66 0:02 +4.91 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rc6 4.Bc1 Be6 5.Ne7 Rc7 6.N7xf5 Bxf5 7.Nxf5 Kxf4 8.Nd6 e3 9.Rxe3 b5 10.Re4+ Kf3 11.Kb2 Rc6 12.Re6 Kf4 13.Rg6 Kf3 14.Rf6+ (48.878.921) 20799
24/66 0:03 +5.11++ 1.Nd5 (71.817.336) 21342
24/66 0:03 +5.08 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rc6 4.Bc1 b5 5.Rf1+ Ke2 6.Rg1 Kf3 7.Rg5 Rh6 8.Kb2 Be6 9.Nxf5 Bxf5 10.Rxf5 Kg3 11.Re5 Kxh4 12.f5 Rh8 13.f6 Rf8 14.Rxe4+ (81.433.172) 21463
25/66 0:04 +5.10 1.Nd5 Re6 2.Re1 Kf3 3.Nge3 Rc6 4.Bc1 b5 5.Rf1+ Ke2 6.Rg1 Kf3 7.Rg5 Rh6 8.Kb2 Be6 9.Nxf5 Bxf5 10.Rxf5 Kg3 11.Re5 Kxh4 12.f5 Rh8 13.f6 Rf8 14.Rxe4+ (101.239.703) 21623
25/66 0:05 +5.29++ 1.Kc1 (110.667.309) 21763
25/66 0:05 +5.71++ 1.Kc1 (113.922.634) 21824
other best move - reduce depth to 22
22/66 0:05 +6.84++ 1.Kc1 (115.714.285) 21837
22/66 0:05 +10.10++ 1.Kc1 (120.691.498) 21947
22/70 0:34 +12.62 1.Kc1 Re5 2.fxe5 Bc6 3.Nxa6 f4 4.Nxf4 Kxf4 5.e6 Ke5 6.e7 Kd6 7.Rxe4 Be8 8.Rxc4 Kxe7 9.Re4+ Kf8 10.c4 Bg6 11.Rd4 Ke7 12.Nc7 Bf5 13.Rf4 Bg4 14.Nd5+ (932.000.708) 26802
23/70 0:35 +13.14++ 1.Kc1 (938.167.795) 26736
23/70 0:35 +14.19++ 1.Kc1 (943.801.010) 26707
23/70 0:35 +17.22++ 1.Kc1 (945.185.019) 26696
23/70 0:35 +M5 1.Kc1 Re5 2.Kd1 Bc8 3.Ke1 a5 4.Kf2 axb4 5.Rg3+ (945.451.784) 26694
24/70 0:35 +M5 1.Kc1 Re5 2.Kd1 Bc8 3.Ke1 a5 4.Kf2 axb4 5.Rg3+ (945.465.429) 26692
25/70 0:35 +M5 1.Kc1 Re5 2.Kd1 Bc8 3.Ke1 a5 4.Kf2 axb4 5.Rg3+ (945.478.745) 26690
26/70 0:35 +M5 1.Kc1 Re5 2.Kd1 Bc8 3.Ke1 a5 4.Kf2 axb4 5.Rg3+ (945.500.278) 26690

Engine: Stockfish 280415 64 BMI2 (8192 MB)
by Tord Romstad, Marco Costalba and Joona
<snip>
37/50 1:59 +6.69++ 1.Nd5 (2.207.575.431) 18424
37/50 1:59 +7.39++ 1.Nd5 (2.208.405.321) 18424
37/50 1:59 +8.44++ 1.Nd5 (2.210.381.019) 18423
37/50 2:00 +10.01++ 1.Nd5 (2.213.533.217) 18423
37/50 2:00 +12.37++ 1.Nd5 (2.221.207.392) 18425
37/50 2:00 +15.91++ 1.Nd5 (2.227.554.098) 18424
37/50 2:01 +21.22++ 1.Nd5 (2.230.281.490) 18424
37/50 2:01 +29.18++ 1.Nd5 (2.230.508.269) 18425
37/50 2:01 +41.11++ 1.Nd5 (2.230.780.538) 18425
37/50 2:01 +59.01++ 1.Nd5 (2.230.952.663) 18425
37/50 2:01 +85.86++ 1.Nd5 (2.231.163.932) 18425
37/50 2:01 +M7 1.Nd5 Re6 2.Kc1 a5 3.Nf6+ Rxf6 4.Kd1 Rg6 5.Ke1 Rg5 6.Kf2 axb4 7.Rg3+ (2.231.354.125) 18426
38/50 2:01 +M6 1.Kc1 Rf7 2.Nd5 Be6 3.Kd1 Bd7 4.Ke1 Rf8 5.Kf2 Ba4 6.Rg3+ (2.232.084.516) 18427
39/50 2:01 +M5 1.Kc1 Rf7 2.Kd1 b5 3.Ke1 Be8 4.Kf2 Rxc7 5.Rg3+ (2.232.177.282) 18427
40/50 2:01 +M5 1.Kc1 Rf7 2.Kd1 b5 3.Ke1 Be8 4.Kf2 Rxc7 5.Rg3+ (2.232.216.646) 18427

[zullil]

Louis amazing test position because it is so few moves.

[d] 8/2Nbr3/pp6/5p1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - -

I simply modified Joseph's original position, to make it more difficult for engines to find a move that wins.