Mate Test Suite

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Eelco de Groot
Posts: 4724
Joined: Sun Mar 12, 2006 2:40 am
Full name:   Eelco de Groot

Re: Mate Test Suite

Post by Eelco de Groot »

Eelco de Groot wrote:
Eelco de Groot wrote: You have no control over nullmove with this, because that is a separate reduction and if nullmove is succesful you only can check it with the verification search which can be extremely shortened.
From a general search point of view, I'm keeping fingers crossed for this patch from Jörg. It just might pass! But then it still would have to go to LTC.

No null-move during verification search. sprt @ 15+0.05 th 1

LLR: 0.25 (-2.94,2.94) [-1.50,4.50]
Total: 3470 W: 668 L: 650 D: 2152

LLR: 0.33 (-2.94,2.94) [-1.50,4.50]
Total: 6048 W: 1162 L: 1134 D: 3752

LLR: 0.93 (-2.94,2.94) [-1.50,4.50]
Total: 9396 W: 1799 L: 1741 D: 5856

LLR: 0.83 (-2.94,2.94) [-1.50,4.50]
Total: 9684 W: 1846 L: 1791 D: 6047

LLR: 1.42 (-2.94,2.94) [-1.50,4.50]
Total: 13684 W: 2632 L: 2545 D: 8507
LLR: 2.95 (-2.94,2.94) [-1.50,4.50]
Total: 27714 W: 5347 L: 5168 D: 17199


It passed STC! Congrats to Jörg. This should be applicable to every chessprogram using nullmove, which we can assume is almost 100% of reasonably developed efforts (but not all use a verification search). But it has to scale which we don't know because, at STC, verification is tested but in a limited way because of the starting depth.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan
Ferdy
Posts: 4856
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Mate Test Suite

Post by Ferdy »

jhellis3 wrote:

Code: Select all

#5	8/2Nb4/pp6/4rp1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - -	Kc1
#5	8/8/2B1N3/3rp3/4k2K/7Q/2r3Pn/1b1N4 w - -   	Kh5
#5	qb6/3N2p1/r2p4/pP1k2p1/1pp3R1/6BB/2P1P3/2N1K3 w - -  	Bh4
#6	1K1N1b2/RPp1pr2/1kP5/2p5/P7/4B1P1/4p1b1/6n1 w - - 	Kc8
#6	n1N3br/2p1Bpkr/1pP2R1b/pP1p1PpR/Pp4P1/1P6/1K1P4/8 w - - 	Rh1
#6	1N1K1b1r/P3pPp1/4k1P1/rp1pB1RN/q4RP1/8/p2pB1p1/1b6 w - - 	Nd7
#6	5R2/2P2pK1/2P2P2/1Pp1BP1P/b6p/1p1RPB2/1p2NPn1/6rk w - - 	Ra8
#7	3K4/1p1B4/bB1k4/rpR1p3/2ppppp1/8/RPPPPP2/r1n5 w - - 	b4
#11	4k1r1/2pnp3/2B1N3/8/8/5R2/6P1/5K2 w - - 	Rf2 or Ke1
#13	n2Bqk2/5p1p/Q4KP1/p7/8/8/8/8 w - - 	Qc8
#16	8/5K2/3p3p/3p3P/pp1P4/rkb1R3/p1p3P1/n1B2B2 w - - 	Rd3
#17	1kn5/p2p4/P1pP1p1q/1PP2P1P/5p2/4rQp1/K7/5B2 w - - 	bxc6
#20	8/6p1/p7/rp1K2p1/kb3pP1/2p2p1b/P1Np1P2/3N2R1 w - - 	Ke6
#21	8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - 	Bh8
#22	1B3N2/5p1B/2K2pn1/5krb/4p1p1/4P1P1/2P1PP2/8 w - - 	Ba7
#27	1k3b1q/pP2p1p1/P1K1P1Pp/7P/2B5/8/8/8 w - - 	Bb5 or Kd5
#29	4rk2/2P2p2/p4P2/2p2b2/2p5/8/P7/2KR4 w - - 	Rd8
I'd be curious to know how various engines perform on the above positions.

Tests should be done single-threaded with 512MB Hash.

I generally only allow 180s max per position to solve, but if you want to allow more that is fine.
I have this engine designed to play positional chess below 2000. No futility pruning, no null move pruning, no LMR, no LMP, routines are not optimized, will not search capture moves in qsearch unless it is winning capture. It got 9/17 at 30s/pos.

Code: Select all

Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Physical Cores: 4
Hyper-Threading: ON
Hash: 128, Threads: 1, Time: 30s/pos
Test file: joseph_ellis_mates.epd
id name CDrill 1700, TotalPos: 17, Correct: 9 (52.94%)

Pos  Correct  EngineBM  ScoreCP  D2Mate   epd
 1        1      b1c1     +894       +5   8/2Nb4/pp6/4rp1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - - bm Kc1; dm 5; id "je 1"; c0 "b1c1;
 2        1      h4h5    +1692       +5   8/8/2B1N3/3rp3/4k2K/7Q/2r3Pn/1b1N4 w - - bm Kh5; dm 5; id "je 2"; c0 "h4h5";
 3        1      g3h4     +441       +5   qb6/3N2p1/r2p4/pP1k2p1/1pp3R1/6BB/2P1P3/2N1K3 w - - bm Bh4; dm 5; id "je 3"; c0 "g3h4";
 4        1      b8c8     +875       +6   1K1N1b2/RPp1pr2/1kP5/2p5/P7/4B1P1/4p1b1/6n1 w - - bm Kc8; dm 6; id "je 4"; c0 "b8c8";
 5        1      h5h1     +288       +6   n1N3br/2p1Bpkr/1pP2R1b/pP1p1PpR/Pp4P1/1P6/1K1P4/8 w - - bm Rh1; dm 6; id "je 5"; c0 "h5h1";
 6        0      f4a4     +406    +1000   1N1K1b1r/P3pPp1/4k1P1/rp1pB1RN/q4RP1/8/p2pB1p1/1b6 w - - bm Nd7; dm 6; id "je 6"; c0 "b8d7";
 7        0     c7c8q    +1911    +1000   5R2/2P2pK1/2P2P2/1Pp1BP1P/b6p/1p1RPB2/1p2NPn1/6rk w - - bm Ra8; dm 6; id "je 7"; c0 "f8a8";
 8        0      a2a5     -110    +1000   3K4/1p1B4/bB1k4/rpR1p3/2ppppp1/8/RPPPPP2/r1n5 w - - bm b4; dm 7; id "je 8"; c0 "b2b4";
 9        0      g2g4     +668    +1000   4k1r1/2pnp3/2B1N3/8/8/5R2/6P1/5K2 w - - bm Rf2 Ke1; dm 11; id "je 9"; c0 "f3f2 f1e1";
10        1      a6c8     +161    +1000   n2Bqk2/5p1p/Q4KP1/p7/8/8/8/8 w - - bm Qc8; dm 13; id "je 10"; c0 "a6c8";
11        0      g2g3       +0    +1000   8/5K2/3p3p/3p3P/pp1P4/rkb1R3/p1p3P1/n1B2B2 w - - bm Rd3; dm 16; id "je 11"; c0 "e3d3";
12        1      b5c6     -301    +1000   1kn5/p2p4/P1pP1p1q/1PP2P1P/5p2/4rQp1/K7/5B2 w - - bm bxc6; dm 17; id "je 12"; c0 "b5c6";
13        0      d5e4       +0    +1000   8/6p1/p7/rp1K2p1/kb3pP1/2p2p1b/P1Np1P2/3N2R1 w - - bm Ke6; dm 20; id "je 13"; c0 "d5e6";
14        0      d4g7     +428       +8   8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; dm 21; id "je 14"; c0 "d4h8";
15        1      b8a7     +167    +1000   1B3N2/5p1B/2K2pn1/5krb/4p1p1/4P1P1/2P1PP2/8 w - - bm Ba7; dm 22; id "je 15"; c0 "b8a7";
16        1      c4b5     +466    +1000   1k3b1q/pP2p1p1/P1K1P1Pp/7P/2B5/8/8/8 w - - bm Bb5 Kd5; dm 27; id "je 16"; c0 "c4b5 c6d5";
17        0      d1d5       +2    +1000   4rk2/2P2p2/p4P2/2p2b2/2p5/8/P7/2KR4 w - - bm Rd8; dm 29; id "je 17"; c0 "d1d8";
There was a surprise move at Pos14.
[d]8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; dm 21; id "je 14"; c0 "d4h8";
With shorter mate in 8 found quickly instead of 21.

Code: Select all

 5/13	00:00	 6k	631k	 0.00	1.Kb4
 6/14	00:00	 20k	619k	 0.00	1.Kb4
 7/16	00:00	 31k	614k	+0.30	1.Kb4
 7/15	00:00	 59k	635k	+0.74	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Nc4 4.Nxc4 Kb1 5.Nxa3+ Ka1
 8/18	00:00	 165k	774k	+0.62	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Na4 4.Kxa4 Kb2 5.Bxa3+ Kc2 6.Nb3
 9/20	00:00	 422k	757k	+2.89	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 Kb1 5.Nxa3+ Kc1 6.Kxa2 d2
 10/20	00:01	 1,015k	810k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 11/21	00:02	 2,386k	817k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 12/23	00:05	 5,397k	903k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
 13/26	00:12	 11,923k	944k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
User avatar
F.Huber
Posts: 894
Joined: Thu Mar 09, 2006 4:50 pm
Location: Austria
Full name: Franz Huber

Re: Mate Test Suite

Post by F.Huber »

Ferdy wrote: There was a surprise move at Pos14.
[d]8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; dm 21; id "je 14"; c0 "d4h8";
With shorter mate in 8 found quickly instead of 21.

Code: Select all

 5/13	00:00	 6k	631k	 0.00	1.Kb4
 6/14	00:00	 20k	619k	 0.00	1.Kb4
 7/16	00:00	 31k	614k	+0.30	1.Kb4
 7/15	00:00	 59k	635k	+0.74	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Nc4 4.Nxc4 Kb1 5.Nxa3+ Ka1
 8/18	00:00	 165k	774k	+0.62	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Na4 4.Kxa4 Kb2 5.Bxa3+ Kc2 6.Nb3
 9/20	00:00	 422k	757k	+2.89	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 Kb1 5.Nxa3+ Kc1 6.Kxa2 d2
 10/20	00:01	 1,015k	810k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 11/21	00:02	 2,386k	817k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 12/23	00:05	 5,397k	903k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
 13/26	00:12	 11,923k	944k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
Maybe shorter, but wrong. ;)

The wrong (i.e. sub-optimal) move of black is 4... d2

Franz
Ferdy
Posts: 4856
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Mate Test Suite

Post by Ferdy »

F.Huber wrote:
Ferdy wrote: There was a surprise move at Pos14.
[d]8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; dm 21; id "je 14"; c0 "d4h8";
With shorter mate in 8 found quickly instead of 21.

Code: Select all

 5/13	00:00	 6k	631k	 0.00	1.Kb4
 6/14	00:00	 20k	619k	 0.00	1.Kb4
 7/16	00:00	 31k	614k	+0.30	1.Kb4
 7/15	00:00	 59k	635k	+0.74	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Nc4 4.Nxc4 Kb1 5.Nxa3+ Ka1
 8/18	00:00	 165k	774k	+0.62	1.Be5 Nd1+ 2.Kb3+ Nb2 3.Bd6 Na4 4.Kxa4 Kb2 5.Bxa3+ Kc2 6.Nb3
 9/20	00:00	 422k	757k	+2.89	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 Kb1 5.Nxa3+ Kc1 6.Kxa2 d2
 10/20	00:01	 1,015k	810k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 11/21	00:02	 2,386k	817k	+4.28	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Bg7+ Kb1 6.Nxd2+ Kc1 7.Nc4 a1Q 8.Bxa1
 12/23	00:05	 5,397k	903k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
 13/26	00:12	 11,923k	944k	+M8	1.Bg7 Nd1+ 2.Kb3+ Nb2 3.Bf8 Nc4 4.Nxc4 d2 5.Kc2 d1Q+ 6.Kxd1 Kb1 7.Nd2+ Kb2 8.Bg7+
Maybe shorter, but wrong. ;)

The wrong (i.e. sub-optimal) move of black is 4... d2

Franz
Tricky indeed, this is why its rating is only 1700 :). But this interests me, I will have a look why it behaves like this.
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: Mate Test Suite

Post by Lyudmil Tsvetkov »

I wonder if the condition the SF programmers are looking for when to disable null move pruning, probcut, etc. is not:

- king can not move
- at least 2 adjacent squares to the king are attacked by enemy pawns or pieces

instead of simply king can not move.

Enemy attacks on adjacent squares are essential with zugzwang positions, but a single attack will not quite suffice, I think they should be at least 2.
lech
Posts: 1175
Joined: Sun Feb 14, 2010 10:02 pm

Re: Mate Test Suite

Post by lech »

In this postion Sting-sf-6 beta; Hash 1024, old dualcore shows other solutuion:
[d]8/7p/6pP/5pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm B?; (17 sec.)

Code: Select all

info depth 33 multipv 1 score cp 969 nodes 144645361 nps 850539 time 170063 pv d4e5 b2d1 c3b3 d1b2 e5d6 b2c4 d2c4 a1b1 d6a3 a2a1n b3c3 b1a2 c4d2 a2a3 d2e4 f5e4
f4f5 g6f5 g5g6 h7g6 h6h7 d3d2 c3d2 a3b4 h7h8q a1b3 d2c2 b3c5 h8g8 f5f4 g8b8 b4c4 b8f4 c4b4 f4b8 b4c4 b8g8 c4b4 g8g6 b4c4 g6g8 c4b5 c2c3 b5c6 c3d4
info depth 34
info nodes 144645361 nps 850539 time 170063
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 981 lowerbound nodes 146541841 nps 853068 time 171782 pv d4e5
info nodes 146541841 nps 853068 time 171782
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 993 lowerbound nodes 152453868 nps 858213 time 177641 pv d4e5
info nodes 152453868 nps 858213 time 177641
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 1012 lowerbound nodes 160960588 nps 864069 time  186282 pv d4e5
info nodes 160960588 nps 864069 time 186282
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 1039 lowerbound nodes 169151548 nps 870161 time  194391 pv d4e5
info nodes 169151548 nps 870161 time 194391
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 1080 lowerbound nodes 179342928 nps 876712 time  204563 pv d4e5
info nodes 179342928 nps 876712 time 204563
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 1141 lowerbound nodes 192627885 nps 884437 time  217797 pv d4e5
info nodes 192627885 nps 884437 time 217797
info currmove d4e5 currmovenumber 1
info depth 34 multipv 1 score cp 1232 lowerbound nodes 212265089 nps 892102 time  237938 pv d4e5
info nodes 212265089 nps 892102 time 237938
info currmove d4e5 currmovenumber 1
info nodes 249309737 nps 893183 time 279125
info currmove c3b3 currmovenumber 2
info currmove d4c5 currmovenumber 3
info currmove c3b4 currmovenumber 4
info currmove d2c4 currmovenumber 5
info nodes 249420069 nps 893127 time 279266
info currmove d2b3 currmovenumber 6
info currmove d2f3 currmovenumber 7
info currmove d4f6 currmovenumber 8
info currmove d4b6 currmovenumber 9
info currmove d4g7 currmovenumber 10
info nodes 251143825 nps 891814 time 281610
info currmove d2f1 currmovenumber 11
info currmove d2b1 currmovenumber 12
info currmove d4h8 currmovenumber 13
info currmove d4a7 currmovenumber 14
info currmove d2e4 currmovenumber 15
info depth 34 multipv 1 score cp 1272 nodes 251387753 nps 891692 time 281922 pv d4e5 b2d1 c3b3 d1b2 e5d6 b2c4 d2c4 a1b1 d6a3 a2a1n b3c3 b1a2 c4d2 a1c2 a3c5 c2e1 d2e4 f5e4 f4f5 d3d2 c3d2 e1f3 d2c3 f3g5 c5e7 g5f7 f5g6 h7g6 h6h7 f7h8 c3d4 a2b3 e7f6 h8f7 d4d5 b3c2 d5e6 f7h6 h7h8q h6f5 e6e5 c2d3 h8a8 d3e3 a8e4 e3f2
To avoid it I added the black pawn a4:
[d]8/7p/6pP/5pP1/p2BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; (3 sec.)

Code: Select all

info depth 24 multipv 1 score cp 0 nodes 813004 nps 285966 time 2843 pv c3b4
info depth 25
info currmove c3b4 currmovenumber 1
info currmove d4e5 currmovenumber 2
info currmove d4f6 currmovenumber 3
info currmove d4g7 currmovenumber 4
info currmove d4h8 currmovenumber 5
info depth 25 multipv 1 score cp 8 lowerbound nodes 1247453 nps 369615 time 3375 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 16 lowerbound nodes 1258108 nps 371123 time 3390 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 28 lowerbound nodes 1278547 nps 373625 time 3422 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 46 lowerbound nodes 1560338 nps 410939 time 3797 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 73 lowerbound nodes 1773416 nps 436586 time 4062 pv d4h8
info nodes 1773416 nps 436586 time 4062
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 114 lowerbound nodes 1899323 nps 450289 time 4218 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 183 lowerbound nodes 2163246 nps 472530 time 4578 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 273 lowerbound nodes 2164966 nps 472906 time 4578 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 404 lowerbound nodes 2167835 nps 471986 time 4593 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 610 lowerbound nodes 2175006 nps 471904 time 4609 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 917 lowerbound nodes 2198871 nps 472266 time 4656 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 25 multipv 1 score cp 1377 lowerbound nodes 2273722 nps 475574 time 4781 pv d4h8
info currmove d4h8 currmovenumber 1
info nodes 2426997 nps 485399 time 5000
info currmove d4e5 currmovenumber 2
info currmove c3b4 currmovenumber 3
info currmove d4f6 currmovenumber 4
info currmove d4g7 currmovenumber 5
info currmove d2f1 currmovenumber 6
info currmove d2c4 currmovenumber 7
info currmove d4c5 currmovenumber 8
info currmove d4a7 currmovenumber 9
info currmove d4b6 currmovenumber 10
info currmove d2f3 currmovenumber 11
info currmove d2b1 currmovenumber 12
info currmove d2b3 currmovenumber 13
info currmove d2e4 currmovenumber 14
info depth 25 multipv 1 score cp 1410 nodes 2434534 nps 483906 time 5031 pv d4h8 b2d1 c3c4 d1b2 c4d4 b2d1 d4d5 d1b2 d5e5 b2c4 e5e6 c4b2 e6f6 b2c4 f6f7 c4b2 f7g7 b2c4 g7h7 c4b2 h7g7 b2d1 g7g6 d1b2 g6f6 b2d1 f6f5 d1b2 f5e5 b2d1 e5e4 d1b2 e4d4 b2d1 d4d3 d1b2 d3d4 b2d1 d4c4 d1b2 c4c3 b2d1 c3b4 d1b2 b4a3
or the pawn a5:
[d]8/7p/6pP/p4pP1/3BpP2/p1KpP3/pn1N4/k7 w - - bm Bh8; (3 sec.)

Code: Select all

info depth 21 multipv 1 score cp 0 nodes 579605 nps 230459 time 2515 pv c3b3 a5a4 b3a3
info depth 22
info currmove c3b3 currmovenumber 1
info currmove d4e5 currmovenumber 2
info currmove d4f6 currmovenumber 3
info currmove d4g7 currmovenumber 4
info currmove d4h8 currmovenumber 5
info depth 22 multipv 1 score cp 48 lowerbound nodes 825701 nps 291973 time 2828 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 19 lowerbound nodes 836417 nps 292555 time 2859 pv d4h8
info nodes 836417 nps 292555 time 2859
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 48 lowerbound nodes 836800 nps 292689 time 2859 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 48 lowerbound nodes 837260 nps 292850 time 2859 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 74 lowerbound nodes 837929 nps 291453 time 2875 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 117 lowerbound nodes 839342 nps 291945 time 2875 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 177 lowerbound nodes 840667 nps 290888 time 2890 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 267 lowerbound nodes 841499 nps 291176 time 2890 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 405 lowerbound nodes 844169 nps 290491 time 2906 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 610 lowerbound nodes 848103 nps 290247 time 2922 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 917 lowerbound nodes 861234 nps 293235 time 2937 pv d4h8
info currmove d4h8 currmovenumber 1
info depth 22 multipv 1 score cp 1410 lowerbound nodes 870086 nps 293155 time 2968 pv d4h8
info currmove d4h8 currmovenumber 1
info currmove c3b3 currmovenumber 2
info currmove d4f6 currmovenumber 3
info currmove d4g7 currmovenumber 4
info currmove d4e5 currmovenumber 5
info currmove d4a7 currmovenumber 6
info currmove d4c5 currmovenumber 7
info currmove d2f1 currmovenumber 8
info currmove d4b6 currmovenumber 9
info currmove d2f3 currmovenumber 10
info currmove d2b1 currmovenumber 11
info currmove d2b3 currmovenumber 12
info currmove d2c4 currmovenumber 13
info currmove d2e4 currmovenumber 14
info depth 22 multipv 1 score cp 1402 nodes 906019 nps 297347 time 3047 pv d4h8 a5a4 c3d4 b2d1 d4d5 d1b2 d5e5 b2d1 e5e6 d1b2 e6f6 b2c4 f6f7 c4b2 f7g7 b2c4 g7h7 c4b2 h7g7 b2d1 g7g6 d1b2 g6f6 b2d1 f6f5 d1b2 f5e5 b2d1 e5e4 d1b2 e4d4 b2d1 d4d3 d1b2 d3c3 b2d1 c3c4 d1b2 c4d4 b2d1
I tried the other tough position from here too:
Sting-sf-6 beta: 1024M Hash, 2 minutes - old dual core
[d]8/5K2/3p3p/3p3P/pp1P4/rkb1R3/p1p3P1/n1B2B2 w - - bm Rd3;

Code: Select all

info depth 39 multipv 1 score cp 0 nodes 90603239 nps 746953 time 121297 pv g2g3
info depth 40
info currmove g2g3 currmovenumber 1
info currmove e3f3 currmovenumber 2
info currmove e3d3 currmovenumber 3
info depth 40 multipv 1 score cp 8 lowerbound nodes 98700624 nps 743854 time 132688 pv e3d3
info nodes 98700624 nps 743854 time 132688
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 16 lowerbound nodes 99343216 nps 743620 time 133594 pv e3d3
info nodes 99343216 nps 743620 time 133594
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 28 lowerbound nodes 99366419 nps 743621 time 133625 pv e3d3
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 46 lowerbound nodes 99475821 nps 743483 time 133797 pv e3d3
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 73 lowerbound nodes 100020958 nps 742959 time 134625 pv e3d3
info nodes 100020958 nps 742871 time 134641
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 114 lowerbound nodes 100311942 nps 742534 time 135094 pv e3d3
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 175 lowerbound nodes 100909421 nps 742319 time 135938 pv e3d3
info nodes 100909421 nps 742319 time 135938
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 267 lowerbound nodes 101521530 nps 742132 time 136797 pv e3d3
info nodes 101521530 nps 742132 time 136797
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 404 lowerbound nodes 103209232 nps 741259 time 139235 pv e3d3
info nodes 103209232 nps 741259 time 139235
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 609 lowerbound nodes 107569683 nps 740899 time 145188 pv e3d3
info nodes 107569683 nps 740899 time 145188
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 916 lowerbound nodes 134140732 nps 734136 time 182719 pv e3d3
info nodes 134140732 nps 734136 time 182719
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 1377 lowerbound nodes 174113971 nps 732580 time  237672 pv e3d3
info nodes 174113971 nps 732580 time 237672
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 2069 lowerbound nodes 174734329 nps 732350 time  238594 pv e3d3
info nodes 174734329 nps 732350 time 238594
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score cp 3106 lowerbound nodes 174765051 nps 732285 time  238657 pv e3d3
info currmove e3d3 currmovenumber 1
info depth 40 multipv 1 score mate 34 lowerbound nodes 174767307 nps 732294 time  238657 pv e3d3
info depth 40 multipv 1 score mate 34 lowerbound nodes 174767307 nps 732248 time  238672 pv e3d3
Maybe, I can't be friendly, but let me be useful.
h1a8
Posts: 518
Joined: Fri Jun 04, 2010 7:23 am

Re: Mate Test Suite

Post by h1a8 »

bob wrote:
Eelco de Groot wrote:I only tried the first mate position so far. Crazy, what I think nullmove does with it :shock:

[d]8/2Nb4/pp6/4rp1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - -

Engine: Rainbow Serpent Sf20150412_019 (512 MB)
by Tord Romstad, Marco Costalba and Joona Kiiski

22/37 0:02 +12.16 1.fxe5 Bc6 2.e6 f4 3.Nxf4 Kxf4 4.e7 Bd7
5.e8Q Bxe8 6.Nxe8 Ke5 7.Ng7 Kf4
8.Nxh5+ Kg4 9.Nf6+ Kxh4 10.Rxe4+ Kg5
11.Nd7 b5 12.Nb8 Kh6 13.Nxa6 Kg5
14.Re5+ (2.473.118) 1144

23/39 0:03 +13.39 1.fxe5 Bc6 2.e6 f4 3.Nxf4 Kxf4 4.e7 Kf5
5.e8Q Bxe8 6.Nxe8 b5 7.Ng7+ Kg4
8.Rxe4+ Kf3 9.Re6 Kg4 10.Rxa6 Kxh4
11.Rg6 Kh3 12.Nxh5 Kh4 13.Ng3 Kh3
14.Nf5 (4.057.998) 1148

.
.
.

26/39 0:12 +14.32 1.fxe5 Bc6 2.e6 f4 3.Nxf4 Kxf4 4.e7 Kf5
5.e8Q Bxe8 6.Nxe8 Kf4 7.Ng7 Ke5
8.Nxh5 Kf5 9.Ng3+ Kf4 10.h5 Kg5
11.Rxe4 Kf6 12.Rxc4 Kg5 13.Rd4 a5
14.b5 (14.375.345) 1178

.
.
.

44/54 45:15 +60.46++ 1.fxe5 (2.889.247.408) 1064

44/54 47:47 +62.03++ 1.fxe5 (3.050.048.216) 1063

44/54 50:37 +64.39++ 1.fxe5 (3.230.026.073) 1063

44/54 52:42 +67.93++ 1.fxe5 (3.361.550.406) 1063

44/54 53:46 +65.28-- 1.fxe5 f4 (3.428.193.135) 1062

44/54 54:10 +69.26++ 1.fxe5 (3.453.667.806) 1062

44/54 55:32 +81.19++ 1.fxe5 (3.539.998.598) 1062

44/54 56:43 +99.09++ 1.fxe5 (3.615.470.913) 1062

44/54 84:20 +85.58-- 1.fxe5 b5 (5.408.159.821) 1068

44/54 88:45 +104.85++1.fxe5 (5.687.937.500) 1068

44/54 99:46 +75.08-- 1.fxe5 Bc6 (6.393.273.220) 1068

44/56 100:32 +99.55++ 1.fxe5 (6.442.427.927) 1067



Now, just try the key move, 1. Kc1, no clearing of hash, and three best moves.

[d]8/2Nb4/pp6/4rp1p/1Pp1pPkP/PpPpR3/1B1P2N1/2K5 b - -

Engine: Sf20150412 (512 MB)
by Tord Romstad, Marco Costalba and Joona Kiiski

31 5:46 -5.96 1...Re7 2.Nd5 Re6 3.Re1 Kf3 4.Nge3 Rd6
5.Rf1+ Ke2 6.Rg1 b5 7.Rg5 Kf3 8.Ne7 Rf6
9.Rxh5 Rb6 10.N7xf5 Rg6 11.Rg5 Bxf5
12.Nxf5 Rf6 13.Ne3 Rxf4 14.Rf5 Rxf5
(344.912.107) 994

31 5:46 -6.42 1...Re6 2.Nxe6 Bxe6 3.a4 Bd7 4.Re1 Kg3
5.Ne3 b5 6.a5 Kxf4 7.Rg1 Ke5 8.Rg5 Be6
9.Rxh5 Kf6 10.Rg5 Bc8 11.Nd5+ Ke5
12.Nc7 Kf4 13.Rg8 Bb7 14.Rb8 Bc6
(344.912.107) 994

31 5:46 -6.80 1...Re8 2.Nxe8 Bxe8 3.Re1 b5 4.Ne3+ Kxf4
5.Rf1+ Kg3 6.Nxf5+ Kg4 7.Nd6 Bc6
8.Rf6 Bd5 9.a4 Kxh4 10.Rf5 Bc6
11.Rc5 e3 12.dxe3 Bf3 13.axb5 axb5
14.Rxb5 Kg3
(344.912.107) 994
_____________________________________________________________

32 11:16 -6.09 1...Re7 2.Nd5 Re6 3.Re1 Kf3 4.Nge3 Rd6
5.Rf1+ Ke2 6.Rg1 Kf3 7.Ne7 b5 8.Rg5 Rf6
9.Rxh5 Rd6 10.N3xf5 Bxf5 11.Nxf5 Rf6
12.Ne3 Rxf4 13.Rf5 Rxf5 14.Nxf5 Kg4
(665.957.198) 983

32 11:16 -6.52 1...Re6 2.Nxe6 Bxe6 3.a4 Bd5 4.Re1 Be6
5.Rf1 Kg3 6.Ne3 b5 7.a5 Bc8 8.Rg1+ Kxf4
9.Rg8 Bd7 10.Rg5 Be6 11.Rxh5 Ke5
12.Rg5 Kf6 13.Ba3 Bc8 14.Nd5+ Ke5
(665.957.198) 983

32 11:16 -7.05 1...Re8 2.Nxe8 Bxe8 3.Re1 b5 4.Ne3+ Kxf4
5.Rf1+ Kg3 6.Nxf5+ Kg4 7.Nd6 Bd7
8.Rf7 Be6 9.Rf6 Bd7 10.Nxe4 Bf5
11.Nd6 Bd7 12.Rf7 Be6 13.Ra7 Kxh4
14.Rxa6 Kg5
(665.957.198) 983

Only after choosing a move for Black and doing another King move it is immediately seen, and on the Shredder chessboard square e1 I thought I saw gets a mate in three score projected on it. But before that, nothing.


[FEN "8/2Nb4/pp6/4rp1p/1Pp1pPkP/PpPpR3/1B1P2N1/1K6 w - -"]

1. Kc1 Re7 {or any other move} 2. Kd1 *


23 0:01 -M3 2...Bb5 3.Ke1 Rxc7 4.Kf2 Rc5 5.Rg3+ (1.592.375) 1042
23 0:01 -M3 2...Be8 3.Ke1 Re6 4.Kf2 b5 5.Rg3+ (1.592.375) 1042
23 0:01 -M3 2...Bc6 3.Ke1 Rxc7 4.Kf2 b5 5.Rg3+ (1.592.375) 1042
This is simply zugzwang city. I let Crafty search for 3 minutes, no mate. I played the king move and it instantly found it was being mated in 4 moves...

It's a known problem with null-move search, but certainly one not particularly worth fixing as it will cost a bunch of Elo in other positions, just to pick up an obscure mate in this position...

There are hundreds of these positions floating around.
the challenge is to find an innovative and effective way to detect zugzwang without costing elo, actually improving it. I believe it is possible.
fishpov
Posts: 115
Joined: Sat Mar 07, 2015 6:05 pm

Re: Mate Test Suite

Post by fishpov »

When we will have the pleasure to test your Sting-Sf-6 ? :)
Thanks.
lech
Posts: 1175
Joined: Sun Feb 14, 2010 10:02 pm

Re: Mate Test Suite

Post by lech »

fishpov wrote:When we will have the pleasure to test your Sting-Sf-6 ? :)
Thanks.
I am working at Sting-sf-6.

Now Sting can be the first quasi-intelligent engine.
I added to old tools:
long sacrifice
zugzwang delayed by pawn moves
zugzwang in draw position
trapped pieces
fortress sacrifice
new tools:
walking (or lazzy) King
divided (by pawns) chessboard
mate as only a win

And I think they are complete tools to avoid the horizon effect.

But, there is a bug (maybe negative feature) in alpha-beta search and I try to find it.
Maybe, I can't be friendly, but let me be useful.
Vinvin
Posts: 5335
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Mate Test Suite

Post by Vinvin »

lech wrote:
fishpov wrote:When we will have the pleasure to test your Sting-Sf-6 ? :)
Thanks.
I am working at Sting-sf-6.

Now Sting can be the first quasi-intelligent engine.
I added to old tools:
long sacrifice
zugzwang delayed by pawn moves
zugzwang in draw position
trapped pieces
fortress sacrifice
new tools:
walking (or lazzy) King
divided (by pawns) chessboard
mate as only a win

And I think they are complete tools to avoid the horizon effect.

But, there is a bug (maybe negative feature) in alpha-beta search and I try to find it.
That's a dream ! 8-)
It's an "expert system" to play chess.
That kills all hopes from humans to play better than computer in whichever existing positions :!:
Great work, Marek :!:


BTW, yesterday, I was looking at on-line automatic analysis and SF still giving a very wrong eval (> 2.00) to this drawish positions in the Hammer - Vachier-Lagrave game :

[d]4R1r1/3P1pk1/5b1p/3K4/3p1PPp/8/8/5B2 w - - 13 43

[pgn]
[Event "3rd Norway Chess 2015"]
[Site "Stavanger NOR"]
[Date "2015.06.17"]
[Round "2"]
[White "Hammer,J"]
[Black "Vachier Lagrave,M"]
[Result "1/2-1/2"]
[WhiteElo "2677"]
[BlackElo "2723"]
[EventDate "2015.06.17"]
[ECO "A29"]

1. c4 Nf6 2. Nc3 e5 3. Nf3 Nc6 4. g3 g6 5. Bg2 Bg7 6. O-O O-O 7. e4 d6 8.
d3 h6 9. Be3 Kh7 10. h3 Nd7 11. Rc1 Nd4 12. Bxd4 exd4 13. Ne2 c5 14. h4 b5
15. cxb5 Rb8 16. a4 a6 17. b4 axb5 18. bxc5 dxc5 19. axb5 Rxb5 20. Nf4 Ne5
21. Bh3 Bb7 22. h5 Nxf3+ 23. Qxf3 Qd6 24. Rb1 Rb4 25. Nd5 Bxd5 26. exd5 c4
27. Rxb4 Qxb4 28. dxc4 Qxc4 29. d6 Qc5 30. d7 Qxh5 31. Qxh5 gxh5 32. Bf5+
Kg8 33. Re1 Bf6 34. Re8 h4 35. g4 Kg7 36. f4 Bd8 37. Kg2 Bf6 38. Bd3 Bd8
39. Kf3 Bf6 40. Bf1 Bd8 41. Ke4 Bf6 42. Kd5 Rg8 43. Kc6 Bd8 44. Rxd8 Rxd8
45. Kc7 Rf8 46. d8=Q Rxd8 47. Kxd8 h5 48. gxh5 Kh6 49. Ke7 Kxh5 50. Be2+
Kh6 51. Bf1 Kh5 52. f5 Kg4 53. f6 h3 54. Bxh3+ Kxh3 55. Kxf7 d3 56. Kg7 d2
57. f7 d1=Q 58. f8=Q Qg4+ 59. Kf7 Qf5+ 60. Ke7 Qxf8+ 61. Kxf8 1/2-1/2
[/pgn]
I'd sacrifice 100 Elo points to get more accurate eval in all positions !