Test position for amateur engines!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Test position for amateur engines!

Post by Edsel Apostol »

[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36

My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?

How fast do other amateur engines could avoid Qxf7?
Top
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: Test position for amateur engines!

Post by michiguel »

Edsel Apostol wrote:[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36

My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?

How fast do other amateur engines could avoid Qxf7?
Gaviota avoids Qxf7 all the time.

I think it is more interesting to see how long does it take to find Rf5 after Qxf7. It is not as "instantaneously" obvious as it seems. The engine should understand that after 1. Qxf7 Rf5 2.Qa7 Re5 3. Rxe5 Qxe5 White is lost.

[D]8/5Qpk/7p/r7/P7/6Pb/1q3B1P/4R1K1 b - - 0 37 bm Rxf5

2.7 for gaviota to see Rf5.

Code: Select all

        15   1       0.0    -8.12  Qxf2+ 20.Qxf2 Rxa4
        24   1       0.0    +0.30  Rxa4
       103   2       0.0    +0.25  Rxa4 20.Qd5
       967   3       0.0    +0.32  Rxa4 20.Qd5 Qc3
      2609   4       0.0    +0.30  Rxa4 20.Qd5 Ra1 21.Qe4+ Kg8 22.Qe8+ Kh7
      7009   5       0.0    +0.28  Rxa4 20.Qd5 Qc3 21.Kh1 Ra5
     69479   6       0.2    +0.36  Rxa4 20.Qd5 Ra5 21.Qe4+ Bf5 22.Qf4 Qb3
     81332   6:      0.3    +0.36  Rxa4 20.Qd5 Ra5 21.Qe4+ Bf5 22.Qf4 Qb3
    292206   7       0.9    +0.49  Rxa4 20.Qd5 Qc2 21.Qh5 Ra2 22.Qf7 Rb2
    400037   7:      1.2    +0.49  Rxa4 20.Qd5 Qc2 21.Qh5 Ra2 22.Qf7 Rb2
    868045   8       2.4    +0.50  Rxa4 20.Qf3 Qc2 21.Qe2 Qc6 22.Be3 Re4
                                   23.Ra1 Re5 24.Qd3+ Kg8
    994912   8       2.7      :-)  Rf5
   1070470   8       2.9      :-)  Rf5
   1203799   8       3.3      :-)  Rf5
   1348001   8       3.7    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   1468997   8:      4.0    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   1628634   9       4.5      :-(  Rf5
   1958256   9       5.4      :-(  
   2073021   9       5.7    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   2077212   9:      5.7    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   2451486  10       6.7    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   3412855  10:      9.2    +5.82  Rf5 20.Qa7 Re5 21.Rf1 Qe2 22.Rb1 Qf3
                                   23.Qb7 Re1+ 24.Rxe1 Qxb7
   5476519  11      14.5      :-)  Rf5
   8004629  11      20.8    +6.82  Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.h4 Qb4
                                   23.Ra1 Bf3 24.Kh2 Qf4+ 25.Kg1 Bc6
  11430232  11:     30.0    +6.82  Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.h4 Qb4
                                   23.Ra1 Bf3 24.Kh2 Qf4+ 25.Kg1 Bc6
  15209367  12      39.8    +7.10  Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.Re3
                                   Qa1+ 23.Be1 Qxa4 24.Bc3 Qa7 25.Kf2 g5
                                   26.Bf6 Qa2+ 27.Kg1
  28335584  12:     73.9    +7.10  Rf5 20.Qxf5+ Bxf5 21.g4 Bxg4 22.Re3
                                   Qa1+ 23.Be1 Qxa4 24.Bc3 Qa7 25.Kf2 g5
                                   26.Bf6 Qa2+ 27.Kg1
Top
Will Singleton
Posts: 128
Joined: Thu Mar 09, 2006 5:14 pm
Location: Los Angeles, CA

Re: Test position for amateur engines!

Post by Will Singleton »

Amateur likewise avoids Qxf7, then sees Rf5 in about 5 sec:

Code: Select all

11 8 139 1621561 Rxa4 Qd5 Qf6 
12 10 385 4563078 Rxa4 Qf3 
12 522 459 5391964 Rf5 Qa7 Re5 Rd1 
13 702 776 8920212 Rf5 Qxf5 Bxf5 g4 Bxg4 a5 Qa2 Re7 Bf3 Re3 
14 718 889 10191188 Rf5 Qxf5 Bxf5 g4 Bxg4 h4 
15 721 1081 12509088 Rf5 Qxf5 Bxf5 g4 Bxg4 h4 Qc2 a5 Bh3 Re3 Qc1 Kh2 Qf1 Rg3 Qxf2 Kxh3 Qf5 Kg2 Qxa5
Top
metax
Posts: 344
Joined: Wed Sep 23, 2009 5:56 pm
Location: Germany

Re: Test position for amateur engines!

Post by metax »

The problem in Twisted Logic seems to be that 1...Rf5 is "pruned away" as useless. Even in the position after 1.Qxf7 the engine has no clue and evaluates the position as approximately equal after Rxa4 at depth 18. But in the position after 1.Qxf7 Rf5 the engine realizes that White is losing at depth 11. So with no pruning at all, the problem should be seen at depth 12 from the position after Qxf7 and at depth 13 from the starting position. As it is not seen, the problem seems to be the pruning of Rf5.
To my surprise,. Stockfish has the same problem at the first plies, but avoids Qxf7 at depth 15 after 2 fail-lows at depth 14 and 15.
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Test position for amateur engines!

Post by bob »

Edsel Apostol wrote:[D] 4Q3/5ppk/7p/r7/P7/6Pb/1q3B1P/4R1K1 w - - 3 36

My engine is having a hard time avoiding Qxf7 here. I disabled nullmove and it couldn't avoid it still. What could be the problem?

How fast do other amateur engines could avoid Qxf7?
It takes Crafty almost 1/2 second to discover that this move is bad:

Code: Select all

               12->   0.26  -0.13   1. Qxf7 Rxa4 2. Qf3 Qc2 3. Qe2 Qg6
                                    4. Qd2 Be6 5. Qd6 Re4 6. Be3 Bc4 (s=2)
               13     0.39     -1   1. Qxf7?                          
               13     0.40     -3   1. Qxf7?                          
               13     0.42     -M   1. Qxf7?                          
               13     0.52  -6.98   1. Qxf7 Rf5 2. Qxf5+ Bxf5 3. g4 Bxg4
                                    4. h4 Qa3 5. Re4 Bd1 6. Rc4 Bb3 7.
                                    Rc7 Qxa4
               13     1.28  -1.30   1. Qe4+ f5 2. Qh4 Bg4 3. Qe7 Rxa4 4.
                                    Qe5 Qb7 5. Qe7 Qb3 6. Bc5 Bf3 7. Qd7
                                    Be4
               13     1.76  -1.09   1. Qc6 Qa2 2. Qe4+ f5 3. Qh4 Bg4 4.
                                    Qd8 Qxa4 5. Qd4 Qa3 6. Qc4 Bf3 7. Bd4
                                    Be4
               13     2.12  -1.05   1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
                                    4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Rd4 7.
                                    Bf4
               13->   2.12  -1.05   1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
                                    4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Rd4 7.
                                    Bf4 (s=9)
               14     2.20  -1.05   1. Qe2 Qxe2 2. Rxe2 Rxa4 3. Bb6 Rb4
                                    4. Bc5 Rc4 5. Bd6 Be6 6. Kg2 Kg6 7.
                                    h3 Bd5+ 8. Kh2 (s=8)
When you say you are having a hard time finding that this is bad, what exactly does that mean? a few seconds? Minutes? This is primarily about f2. If you capture on f7, then Rf5 creates a real problem, but it apparently takes a few plies to work out all the hanging pieces.
Top
metax
Posts: 344
Joined: Wed Sep 23, 2009 5:56 pm
Location: Germany

Re: Test position for amateur engines!

Post by metax »

Well, at least on my computer TL 20090922 has a real problem with this position. Even at ply 20 (I know this 'even' probably sounds ridiculous to people with a cluster ;)), which is reached after 4:10 minutes here, it does not see that Qxf7 is very bad. I have aborted the search here, but this means that it would probably play that move in a tournament game.
Top
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Test position for amateur engines!

Post by bob »

metax wrote:Well, at least on my computer TL 20090922 has a real problem with this position. Even at ply 20 (I know this 'even' probably sounds ridiculous to people with a cluster ;)), which is reached after 4:10 minutes here, it does not see that Qxf7 is very bad. I have aborted the search here, but this means that it would probably play that move in a tournament game.
What I usually do here is use a little human chess knowledge and walk down the PV a couple of moves (down the PV it thinks is best) and then see why it can't see the problem that is brewing around f2. You want a small tree to debug, not a huge one.

Here's an idea, assuming you can run Crafty, which spots this at 12 ply if I remember the output I posted.

Run down the PV for 4 full moves, and then feed the resulting position to Crafty (set noise to 0 so that you can see all output). Limit Crafty to something shallow using the sd=n command so that the tree will be very small. Keep walking down the PV until Crafty suddenly fails low (you have already played Qxf7 here so it is going to fail low eventually).

Now take that same position and feed it to your program and see how long it takes it to fail low. You will have a very simple PV output from Crafty, so you should be able to figure out what you are missing when you see your search output.

The idea is to keep shrinking the tree until you are debugging a 2-3-4 ply search which is very easy to work with, rather than a 20 ply search which is very difficult to debug.
Top
Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Re: Test position for amateur engines!

Post by Howard E »

ProDeo Hera: core i7
10 00:00 1.374.429 1.374.429 -0.24 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qb1+ g6
11 00:01 2.774.058 2.774.058 -4.53 Qxf7 Rf5 Qa7 Re5 Rxe5 Qxe5 Qd4 Qe2 Qa1 Qf3 Qf1 Bxf1 Kxf1 11 00:02 4.926.565 2.463.282 -0.69 Re2 Qb3 Re3 Qxa4 Qxa4 Rxa4 Re7 Ra1+ Re1 Ra4 Bb6 Rb4 Bc5 Rb2 Re8 Kg6 Re7
11 00:02 6.222.010 3.111.005 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Bf5 Kf2 Kg6 Ke3 Ra1 Bd4 Rb1
12 00:04 9.627.058 2.259.872 -0.62 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Be6 Kf2 Rb4 Ba5 Rb1 Rc2 Ra1 Bb6 Bb3 Kf3
13 00:05 11.472.751 2.181.131 -0.61 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rc4 Bd6 Rd4 Bc5 Rd5 Bb4 Be6
14 00:06 16.167.164 2.391.592 -0.59 Qe2 Qxe2 Rxe2 Rxa4 Bb6 Rb4 Bc5 Rb1+ Kf2 Kg6 Kf3 Rd1 Re7 Rd2 Bf2 Be6
15 00:11 28.840.595 2.403.382 -0.64 Qe2 Qb3 Qe4+ Bf5 Qf4 Qxa4 Qxa4 Rxa4 Re7 Ra2 Rxf7 Ra1+ Kg2 Bg6 Ra7 Be4+ Kh3 h5
Top
Michel
Posts: 2272
Joined: Mon Sep 29, 2008 1:50 am

Re: Test position for amateur engines!

Post by Michel »

Not a good result for the latest version of GnuChess this time. On a Pentium 4 (which is very slow) GnuChess 5.07.17testb need precisely one hour to avoid Qxf7 at depth 16 (the fail low occurs after 40 minutes).

I wonder how the other engines manage to find it so quickly... It is after all quite
deep and involves a few relatively quiet moves. This version of GnuChess does LMR which is probably the culprit.

I am going to test with the original version of GnuChess which just does a full width search with extensions.
Top
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: Test position for amateur engines!

Post by zamar »

Interesting position! There are silent moves (like Re5) which can get pruned away.

Here is how current Stockfish development version survives here. On depth 12 it sees that sth is badly wrong, but because Stockfish doesn't resolve fail lows immediately, it sees the way out on depth 13.

2 -0.12 00:00 259 Qxf7 Rxa4
3 -0.16 00:00 778 Qxf7 Rxa4 Qf3
4 -0.28 00:00 1585 Qxf7 Rxa4 Re8 Ra1+ Re1
5 -1.01 00:00 4207 Qxf7 Rxa4 Qh5 Ra2 Qf3
5 -0.08 00:00 6791 Qe4+ g6 Bd4 Qa2 Re2 Qxa4
6 -0.48 00:00 14553 Qe4+ f5 Qc6 Qb4 Ra1 Qe4 Qxe4 fxe4
7 -0.61 00:00 22240 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qxa4 Rxa4 Bc5 Bg4
8 -0.53 00:00 38200 Qe4+ f5 Qc6 Qa2 Bd4 Qxa4 Qb6 Qc2 Qb7
9 -0.73 00:00 69700 Qe4+ f5 Qc6 Qa2 Re7 Rxa4 Qc3 Rg4 Qc6 Qb2
9 -0.48 00:00 81692 Qxf7 Rxa4 Qf3 Qc2 Qd1 Ra2 Qxc2+ Rxc2 Bd4 Kg6
10 -0.44 00:00 115773 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Kg6 Bd4
11 -0.32 00:00 185087 Qxf7 Rxa4 Qf3 Qc2 Qe2 Ra2 Qxc2+ Rxc2 Re7 Rd2 Be3
Re2
12 < -0.61 00:00 282325 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 < -1.86 00:00 448441 Qxf7 Rf5 Qa7 Re5 Rd1 Qc2 Qd4 Qc6 Qd3+ Rf5
13 -0.97 00:00 1123k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qf4 Be4 g4 Rxa4 gxf5 Qc2
f6 Bd5
14 -1.13 00:01 1846k Qe4+ f5 Qh4 Bg4 h3 Bf3 Qd4 Qxd4 Bxd4 Rxa4 Kf2
Rxd4 Kxf3 Kg6 Re7 Kf6 Ke3 Kxe7 Kxd4
14 -1.05 00:03 4294k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rd2 Re4 Rb2
Rc4 Bd6 Rc6 Bf4
15 -1.01 00:04 4771k Qe2 Qxe2 Rxe2 Rxa4 Bc5 Kg6 Kf2 Bg4 Rb2 Re4 Bd6 f6
Bc5 Kf5 Rd2 Re5
16 -1.17 00:09 11200k Qe2 Qb3 Qe4+ Bf5 Qe3 Qxa4 Qc3 f6 Qd2 Qa2 Qxa2
Rxa2 Kg2 Kg6 Re7 h5 Kf3 Bg4+ Kg2
Joona Kiiski
Top

Return to “Computer Chess Club: General Topics”