Test position for amateur engines!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Eelco de Groot
Posts: 4279
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Test position for amateur engines!

Post by Eelco de Groot » Tue Dec 01, 2009 7:50 pm

That looks interesting Joona! Trying to fully resolve 2.Qa7 is just very hard I think, as -Michel's version of?- GnuChess also found, but if you find a way of just letting 1.Qxf7 "fall out of the search window", you gain time! That is just my interpretation of the output of your development version.

I got this output where I could not resolve 2. Qc4 after 300 minutes. The score for Qa7 is low enough but it takes time to get an accurate enough score and the lower alpha is disastrous for the Null window searches of other moves, because hash results and imprecise moves just are not good enough to fail low anymore. There is also (theoretically at least) the danger of first wrongly failing high with your Null window search, with an inferior Root move, then if your PV search of this Root move is also not good enough to fail low against -9.77, you end up with a new but wrong "best" move.


[FEN "4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - -"]

1. Qxf7 Rf5 *

[D]8/5Qpk/7p/5r2/P7/6Pb/1q3B1P/4R1K1 w - -

Engine: Rainbow Serpent 1.5 Cubic II Build 077 (256 MB)
by Romstad, Costalba, Kiiski, De Groot

1.00 0:00 +1.97 2.Qa7 (309) 0

2.00 0:00 +1.13 2.Qa7 Re5 3.Rd1 (1.255) 3

3.00 0:00 +1.13 2.Qa7 Re5 3.Rd1 (6.137) 15

4.00 0:00 +1.05 2.Qa7 Re5 3.Rd1 Qb4 (12.489) 29

5.00 0:00 0.00 2.Qa7 Re5 3.Rd1 Rd5 4.Re1 Re5 (35.704) 78

6.00 0:00 0.00 2.Qa7 Re5 3.Rd1 Rd5 4.Re1 Re5 (58.753) 117

7.00 0:00 0.00 2.Qa7 Re5 3.Rd1 Rd5 4.Re1 Re5 (95.500) 169

8.01 0:01 0.00 2.Qa7 Re5 3.Rd1 Rd5 4.Re1 Re5 (381.113) 369

9.01 0:01 -0.20-- 2.Qa7 Re5 3.Rd1 Qc2 4.Qd4 Qc6 5.Qd3+ g6 (622.158) 437

10.01 0:06 -1.01-- 2.Qa7 Re5 3.Rd1 Qc2 4.Qd4 Qc6 5.Qd3+ g6 (3.675.124) 590

11.01 0:20 -4.24-- 2.Qa7 Re5 3.Rd1 Qc2 4.Qd4 Qc6 5.Qd3+ g6 (13.584.651) 670

12.01 12:35 -9.77 2.Qa7 Re5 3.Rf1 Bxf1 4.Kxf1 Qb1+
5.Kg2 Qe4+ 6.Kf1 Rd5 7.Qe3 Rd1+
8.Ke2 Qg4+ 9.Qf3 Rd2+ 10.Kxd2 Qxf3
11.Be3 Qg2+ 12.Kc3 Qc6+ 13.Kd2 Qxa4
14.h4 Qa2+ 15.Ke1 (514.329.559) 680

best move: Qf7-a7 time: 304:05.250 min n/s: 585.192 nodes: 10.676.970.309

After 300 minutes still calculating 2. Qc4 (2/38)
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

Michel
Posts: 2210
Joined: Sun Sep 28, 2008 11:50 pm

Re: Test position for amateur engines!

Post by Michel » Tue Dec 01, 2009 7:59 pm

Not a good result for the latest version of GnuChess this time. On a Pentium 4 (which is very slow) GnuChess 5.07.17testb need precisely one hour to avoid Qxf7 at depth 16 (the fail low occurs after 40 minutes).
Well I redid the test with stock GnuChess 5.07. On this computer (a Pentium 4) GnuChess
needs 24653 seconds to fail low and 34622 seconds to resolve the fail low (at depth 11 though).

I tweaked the search algorithm a bit and now my latest version GnuChess 5.07.18testb
avoids Qxf7 after 17 minutes at depth 15. Progress I guess.

Note that GnuChess is handicapped here since it is a bitboard engine. On a core duo it is
much faster.

Edsel Apostol
Posts: 770
Joined: Mon Jul 17, 2006 3:53 am
Full name: Edsel Apostol
Contact:

Re: Test position for amateur engines!

Post by Edsel Apostol » Tue Dec 01, 2009 11:54 pm

zamar wrote:Interesting position! There are silent moves (like Re5) which can get pruned away.

Here is how current Stockfish development version survives here. On depth 12 it sees that sth is badly wrong, but because Stockfish doesn't resolve fail lows immediately, it sees the way out on depth 13.

2 -0.12 00:00 259 Qxf7 Rxa4
3 -0.16 00:00 778 Qxf7 Rxa4 Qf3
4 -0.28 00:00 1585 Qxf7 Rxa4 Re8 Ra1+ Re1
5 -1.01 00:00 4207 Qxf7 Rxa4 Qh5 Ra2 Qf3
5 -0.08 00:00 6791 Qe4+ g6 Bd4 Qa2 Re2 Qxa4
6 -0.48 00:00 14553 Qe4+ f5 Qc6 Qb4 Ra1 Qe4 Qxe4 fxe4
7 -0.61 00:00 22240 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qxa4 Rxa4 Bc5 Bg4
8 -0.53 00:00 38200 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qb6 Qc2 Qb7
9 -0.73 00:00 69700 Qe4+ f5 Qc6 Qa2 Re7 Rxa4 Qc3 Rg4 Qc6 Qb2
9 -0.48 00:00 81692 Qxf7 Rxa4 Qf3 Qc2 Qd1 Ra2 Qxc2+ Rxc2 Bd4 Kg6
10 -0.44 00:00 115773 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Kg6 Bd4
11 -0.32 00:00 185087 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Rd2 Be3
Re2
12 < -0.61 00:00 282325 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 < -1.86 00:00 448441 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 -0.97 00:00 1123k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qf4 Be4 g4 Rxa4 gxf5 Qc2
f6 Bd5
14 -1.13 00:01 1846k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qd4 Qxd4 Bxd4 Rxa4 Kf2
Rxd4 Kxf3 Kg6 Re7 Kf6 Ke3 Kxe7 Kxd4
14 -1.05 00:03 4294k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rd2 Re4 Rb2
Rc4 Bd6 Rc6 Bf4
15 -1.01 00:04 4771k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rb2 Re4 Bd6 f6
Bc5 Kf5 Rd2 Re5
16 -1.17 00:09 11200k Qe2 Qb3 Qe4+ Bf5 Qe3 Qxa4 Qc3 f6 Qd2 Qa2 Qxa2
Rxa2 Kg2 Kg6 Re7 h5 Kf3 Bg4+ Kg2
You are right! My engine couldn't find Re5 that is a crucial move to the main variation. I disabled LMR, Null move, Prunings, and the combination of them and it still couldn't find the right variation.

Even at this position:

[D] 8/Q5pk/7p/5r2/P7/6Pb/1q3B1P/4R1K1 b - - 2 37

it takes depth 16 to find that Re5 is winning.

Even Doch has a hard time with the original position and would certainly miss it also in blitz.

Paloma
Posts: 1010
Joined: Thu Dec 25, 2008 8:07 pm

Re: Test position for amateur engines!

Post by Paloma » Wed Dec 02, 2009 12:42 am

Hi Edsel,
what about the new Twisted Logic?
Is the search issue you reported about [url http://www.talkchess.com/forum/viewtopi ... 62&t=29841] here
[/url] solved?
Thanks

Edsel Apostol
Posts: 770
Joined: Mon Jul 17, 2006 3:53 am
Full name: Edsel Apostol
Contact:

Re: Test position for amateur engines!

Post by Edsel Apostol » Wed Dec 02, 2009 3:32 am

Paloma wrote:Hi Edsel,
what about the new Twisted Logic?
Is the search issue you reported about http://www.talkchess.com/forum/viewtop ... 41] here solved?
Thanks
Hi Herbert,

Yes, I've fixed the search issue. It's concerning some very aggressive reductions that I didn't intend to that leads TL to be weaker at longer time controls.

The latest version still has the same problem though as the last public version on the position I've posted. I'm working on it as of now. I'm going to give it a few days and if I couldn't have a solution after that, I might as well forget it for a while and continue with other ideas I wanted to try.

jdart
Posts: 4103
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Test position for amateur engines!

Post by jdart » Wed Dec 02, 2009 3:48 am

Arasan is not very fast on this but does get it in <1 minute on a quad:
am Qxf7
0 0.01 Qxf7 -0.12 3
0 0.01 Re7 +0.28 29
1 0.01 Re7 +0.16 147
2 0.01 Re7 +0.12 1266
2 0.01 Qxf7 +0.44 1910
3 0.01 Qxf7 +0.00 2810
4 0.03 Qe4+ +0.00 11473
5 0.06 Qe4+ -0.20 64092
6 0.10 Qe4+ -0.04 114825
7 0.12 Qe4+ -0.19 169364
8 0.18 Qe4+ -0.20 293188
8 0.20 Qxf7 -0.19 342455
9 0.32 Qxf7 -0.24 694215
10 0.42 Qxf7 -0.24 954692
11 0.81 Qxf7 -0.31 2058778
12 1.07 Qxf7 -0.28 2939201
13 1.68 Qxf7 -0.27 4777772
14 2.71 Qxf7 -0.24 7995031
15 4.62 Qxf7 -0.31 14227752
16 13.51 -- -0.81 43622263
16 16.35 Qxf7 -5.00 52700893
16 57.50 Re2 -1.28 189478153
3.30M nodes/second.
11 tablebase probes, 11 tablebase hits
969 splits, average thread usage=3.81
result: Re2 score: -1.28 ++ solved in 57.50 sec. (189.48M nodes)
Re2 Qa1+ Re1 Qxa4 Qxa4 Rxa4 Re2 Kg6 Be1 Bg4 Rb2 Bf3 Kf2 Bd5 Bb4 Kf5 Bc5 f6

Post Reply