strange analysis by stockfish1.5

[Uri Blass]

I gave it 256 mbytes hash.

see the pv at depth 21

I wonder how it got
1.Bb7xd5 Ke7-e8 in the pv when the score is only +1.61 pawns for white.

Is it a hash bug?


New game
[D]8/1B2kpp1/p3p2p/3n4/PPp5/7P/2PK1PP1/8 w - - 0 1

Analysis by Stockfish 1.5 JA:


1.Bb7xa6
² (0.32) Depth: 1 00:00:00
1.Bb7xa6 c4-c3+ 2.Kd2-e2 Nd5xb4
² (0.32) Depth: 2 00:00:00
1.b4-b5 c4-c3+ 2.Kd2-d3
² (0.36) Depth: 2 00:00:00
1.c2-c3 Nd5-c7
² (0.60) Depth: 2 00:00:00
1.c2-c3 Nd5-c7 2.Bb7-e4
² (0.64) Depth: 3 00:00:00
1.Bb7xd5 e6xd5 2.b4-b5 a6xb5 3.a4xb5
± (0.84) Depth: 3 00:00:00
1.Bb7xd5 e6xd5 2.b4-b5 a6xb5 3.a4xb5
± (0.84) Depth: 4 00:00:00
1.Bb7xd5 e6xd5 2.c2-c3 Ke7-d6 3.b4-b5 a6xb5 4.a4xb5
² (0.68) Depth: 5 00:00:00
1.Bb7xd5 e6xd5 2.b4-b5 a6xb5 3.a4xb5 Ke7-d6 4.Kd2-e3
± (0.76) Depth: 6 00:00:00 4kN
1.Bb7xd5 e6xd5 2.b4-b5 a6xb5 3.a4xb5 Ke7-d6 4.Kd2-e3 Kd6-c5
² (0.48) Depth: 7 00:00:00 6kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.Bb7-c6 Nc7-e6 4.b4-b5
² (0.56) Depth: 7 00:00:00 9kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.Bb7-c6 Ke7-d6 4.b4-b5 Nc7-e6
² (0.52) Depth: 8 00:00:00 12kN
1.c2-c3 Nd5-c7 2.b4-b5 a6xb5 3.a4-a5 Ke7-d6 4.a5-a6
± (0.80) Depth: 9 00:00:01 26kN
1.c2-c3 Nd5-c7 2.b4-b5 a6xb5 3.a4-a5 Ke7-d6 4.a5-a6 Kd6-c5 5.a6-a7 b5-b4 6.c3xb4+ Kc5xb4 7.Kd2-e3
+- (1.65) Depth: 10 00:00:01 38kN
1.c2-c3 Nd5-c7 2.b4-b5 a6xb5 3.a4-a5 Ke7-d6 4.a5-a6 Kd6-c5 5.a6-a7 b5-b4 6.c3xb4+ Kc5xb4 7.g2-g4 Kb4-c5 8.a7-a8Q Nc7xa8 9.Bb7xa8
± (1.17) Depth: 11 00:00:01 56kN
1.c2-c3 Nd5-c7 2.b4-b5 a6xb5 3.a4-a5 Ke7-d6 4.a5-a6 Kd6-c5 5.a6-a7 b5-b4 6.c3xb4+ Kc5xb4 7.Kd2-e3 Kb4-c5 8.Ke3-e4
+- (1.41) Depth: 12 00:00:01 96kN
1.c2-c3 Nd5-c7 2.b4-b5 a6xb5 3.a4-a5 Ke7-d6 4.a5-a6 Kd6-c5 5.a6-a7 b5-b4 6.c3xb4+ Kc5xb4 7.Bb7-e4 Kb4-c5 8.g2-g3 e6-e5 9.a7-a8Q Nc7xa8 10.Be4xa8
± (1.17) Depth: 13 00:00:01 169kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 b5-b4 6.c3xb4 Nc7-b5 7.Bb7-e4 Nb5-a7 8.g2-g3
+- (1.57) Depth: 14 00:00:01 713kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 b5-b4 6.c3xb4 Nc7-b5 7.Bb7-e4 Nb5-a7 8.g2-g3 c4-c3 9.Ke3-d3
+- (1.61) Depth: 15 00:00:01 910kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 b5-b4 6.c3xb4 Nc7-b5 7.Bb7-e4 h6-h5 8.g2-g3 Nb5-a7 9.h3-h4
+- (1.57) Depth: 16 00:00:02 1288kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 b5-b4 6.c3xb4 Nc7-b5 7.Bb7-e4 g7-g6 8.g2-g3 f7-f5
± (1.37) Depth: 17 00:00:03 1728kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 Kd6-c5 6.a6-a7 b5-b4 7.c3xb4+ Kc5xb4 8.Ke3-e4 c4-c3 9.Ke4-d3 Kb4-c5 10.a7-a8Q Nc7xa8 11.Bb7xa8 Kc5-b4 12.Kd3-c2
+- (1.45) Depth: 18 00:00:03 3364kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.b4-b5 a6xb5 4.a4-a5 Ke7-d6 5.a5-a6 Kd6-c5 6.a6-a7 b5-b4 7.c3xb4+ Kc5xb4 8.Ke3-e4 c4-c3 9.Ke4-d3 f7-f5 10.Kd3-c2 e5-e4 11.g2-g4 g7-g6 12.g4xf5 g6xf5
+- (1.49) Depth: 19 00:00:05 5395kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.f2-f4 Ke7-d6 4.f4xe5+ Kd6xe5 5.b4-b5 a6xb5 6.a4-a5 f7-f5 7.a5-a6 f5-f4+ 8.Ke3-d2 Ke5-d6 9.a6-a7 Kd6-c5 10.a7-a8Q Nc7xa8 11.Bb7xa8 g7-g5 12.Ba8-e4 b5-b4 13.c3xb4+ Kc5xb4
+- (1.41) Depth: 20 00:00:10 10876kN
1.c2-c3 Nd5-c7 2.Kd2-e3 e6-e5 3.f2-f4 Ke7-d6 4.f4xe5+ Kd6xe5 5.b4-b5 a6xb5 6.a4-a5 f7-f5 7.a5-a6 f5-f4+ 8.Ke3-d2 Ke5-d6 9.a6-a7 Kd6-c5 10.a7-a8Q Nc7xa8 11.Bb7xa8 g7-g5 12.Ba8-e4 b5-b4 13.c3xb4+ Kc5xb4 14.Be4-d5
± (1.37) Depth: 21 00:00:19 21995kN
1.Bb7xd5 Ke7-e8 2.Bd5xc4 Ke8-e7 3.Bc4xa6
+- (1.61) Depth: 21 00:00:30 34511kN
1.Bb7xd5 e6xd5 2.Kd2-e3 h6-h5 3.g2-g4 h5xg4 4.h3xg4 Ke7-d6 5.Ke3-d4 Kd6-c6 6.f2-f4 Kc6-d6 7.c2-c3 Kd6-c6 8.f4-f5 g7-g6 9.f5xg6 f7xg6 10.Kd4-e5 g6-g5 11.a4-a5 d5-d4 12.Ke5xd4
+- (1.81) Depth: 22 00:01:01 66705kN
1.Bb7xd5 e6xd5 2.Kd2-e3 h6-h5 3.g2-g4 h5xg4 4.h3xg4 Ke7-d6 5.Ke3-d4 f7-f6 6.f2-f4 Kd6-c6 7.g4-g5 f6xg5 8.f4xg5 Kc6-d6 9.c2-c3 Kd6-c6 10.Kd4-e5 g7-g6 11.a4-a5 d5-d4 12.Ke5xd4 Kc6-b5
+- (1.89) Depth: 23 00:01:39 102833kN
1.Bb7xd5 Ke7-f6 2.Bd5xc4 Kf6-e7 3.Bc4xa6
+- (2.10) Depth: 24 00:02:15 136691kN

(so k, 06.10.2009)

[gladius]

That whole line is really wierd.

1.Bb7xd5 Ke7-e8 2.Bd5xc4 Ke8-e7 3.Bc4xa6

Bishop takes, random king move, bishop takes another pawn, random king move, bishop takes another pawn! Certainly not a 1.6 score . So there must be a hash problem here. I believe the PV comes from the hash, so this might be a bit bigger bug, or it could just be a reconstruction problem.

It also looks like it came after a fail high, so maybe since Bxd5 had failed high, the next move is not correctly stored in the hash table?

[zamar]

Explanation is pretty obvious here:

* New thing added to Stockfish 1.5 is that it _extends_ PV from transposition table. Usually it works, but sometimes it gives bizarre lines like this. Without extension it would only show "PV = 1. Bb7 x d5".

[gladius]

Explanation is pretty obvious here:

* New thing added to Stockfish 1.5 is that it _extends_ PV from transposition table. Usually it works, but sometimes it gives bizarre lines like this. Without extension it would only show "PV = 1. Bb7 x d5".

Hm.. I didn't look at it very long, but it seems like this would be a problem. If the hash entry for the position after Bxd5 has Ke8 as best move, that's not so great for move ordering the next time the position is visited.

[Uri Blass]

Explanation is pretty obvious here:

* New thing added to Stockfish 1.5 is that it _extends_ PV from transposition table. Usually it works, but sometimes it gives bizarre lines like this. Without extension it would only show "PV = 1. Bb7 x d5".

I still do not understand.

How did the tranposition table get Ke8 after Bxd5

Uri

[Tord Romstad]

Explanation is pretty obvious here:

* New thing added to Stockfish 1.5 is that it _extends_ PV from transposition table. Usually it works, but sometimes it gives bizarre lines like this. Without extension it would only show "PV = 1. Bb7 x d5".

I still do not understand.

How did the tranposition table get Ke8 after Bxd5

I agree: It does look a little suspicious in this case. I'll have a look at it and try to find out what really happens, if I'm able to reproduce your analysis (should be easy if you used a single search thread, but perhaps very difficult if you used more than one).

You'll always get a good PV in the end, by waiting until the fail high has been resolved. In practice, this can sometimes take a very long time, because Stockfish doesn't try to resolve fail highs at the root immediately, but postpones it to the next iteration instead (most people would probably say that this is absurd and completely crazy and obviously can't work well, and I would be tempted to agree. Fortunately Joona didn't think so, and to my surprise it seems to work extremely well in practice). Sometimes the search will fail high at the next iteration as well, and you may end up waiting several plies for an exact score and a real PV. I wonder if it would be a good idea to disable aspiration windows in analysis mode, to make sure the user always get exact scores and high-quality PVs.

[Werner]

Hi Tord,
I can repeat it with 256 MB hash (not with 128)

Werner
Neue Partie
8/1B2kpp1/p3p2p/3n4/PPp5/7P/2PK1PP1/8 w - - 0 1

Analysis by Stockfish 1.5 JA 64bit 1C:

1.Lxd5 exd5 2.b5 axb5 3.axb5 Kd6 4.Ke3
+/- (0.76) Tiefe: 6 00:00:00 4kN
1.Lxd5 exd5 2.b5 axb5 3.axb5 Kd6 4.Ke3 Kc5
+/= (0.48) Tiefe: 7 00:00:00 6kN
1.c3 Sc7 2.Ke3 e5 3.Lc6 Se6 4.b5
+/= (0.56) Tiefe: 7 00:00:00 9kN
1.c3 Sc7 2.Ke3 e5 3.Lc6 Kd6 4.b5 Se6
+/= (0.52) Tiefe: 8 00:00:00 12kN
1.c3 Sc7 2.b5 axb5 3.a5 Kd6 4.a6
+/- (0.80 !) Tiefe: 9 00:00:00 26kN
1.c3 Sc7 2.b5 axb5 3.a5 Kd6 4.a6 Kc5 5.a7 b4 6.cxb4+ Kxb4 7.Ke3
+- (1.65) Tiefe: 10 00:00:00 38kN
1.c3 Sc7 2.b5 axb5 3.a5 Kd6 4.a6 Kc5 5.a7 b4 6.cxb4+ Kxb4 7.g4 Kc5 8.a8D Sxa8 9.Lxa8
+/- (1.17) Tiefe: 11 00:00:00 56kN
1.c3 Sc7 2.b5 axb5 3.a5 Kd6 4.a6 Kc5 5.a7 b4 6.cxb4+ Kxb4 7.Ke3 Kc5 8.Ke4
+- (1.41) Tiefe: 12 00:00:00 96kN
1.c3 Sc7 2.b5 axb5 3.a5 Kd6 4.a6 Kc5 5.a7 b4 6.cxb4+ Kxb4 7.Le4 Kc5 8.g3 e5 9.a8D Sxa8 10.Lxa8
+/- (1.17) Tiefe: 13 00:00:00 169kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 b4 6.cxb4 Sb5 7.Le4 Sa7 8.g3
+- (1.57) Tiefe: 14 00:00:00 713kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 b4 6.cxb4 Sb5 7.Le4 Sa7 8.g3 c3 9.Kd3
+- (1.61) Tiefe: 15 00:00:00 910kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 b4 6.cxb4 Sb5 7.Le4 h5 8.g3 Sa7 9.h4
+- (1.57) Tiefe: 16 00:00:01 1288kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 b4 6.cxb4 Sb5 7.Le4 g6 8.g3 f5
+/- (1.37 --) Tiefe: 17 00:00:01 1728kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 Kc5 6.a7 b4 7.cxb4+ Kxb4 8.Ke4 c3 9.Kd3 Kc5 10.a8D Sxa8 11.Lxa8 Kb4 12.Kc2
+- (1.45) Tiefe: 18 00:00:02 3364kN
1.c3 Sc7 2.Ke3 e5 3.b5 axb5 4.a5 Kd6 5.a6 Kc5 6.a7 b4 7.cxb4+ Kxb4 8.Ke4 c3 9.Kd3 f5 10.Kc2 e4 11.g4 g6 12.gxf5 gxf5
+- (1.49) Tiefe: 19 00:00:03 5395kN
1.c3 Sc7 2.Ke3 e5 3.f4 Kd6 4.fxe5+ Kxe5 5.b5 axb5 6.a5 f5 7.a6 f4+ 8.Kd2 Kd6 9.a7 Kc5 10.a8D Sxa8 11.Lxa8 g5 12.Le4 b4 13.cxb4+ Kxb4
+- (1.41) Tiefe: 20 00:00:06 10876kN
1.c3 Sc7 2.Ke3 e5 3.f4 Kd6 4.fxe5+ Kxe5 5.b5 axb5 6.a5 f5 7.a6 f4+ 8.Kd2 Kd6 9.a7 Kc5 10.a8D Sxa8 11.Lxa8 g5 12.Le4 b4 13.cxb4+ Kxb4 14.Ld5
+/- (1.37) Tiefe: 21 00:00:13 21995kN
1.Lxd5 Ke8 2.Lxc4 Ke7 3.Lxa6
+- (1.61 !) Tiefe: 21 00:00:20 34511kN

(, 06.10.2009)

[Tord Romstad]

Hi Tord,
I can repeat it with 256 MB hash (not with 128)

Excellent. I'll have a look at it a little later today, when I've finished testing the Chess960 bug fix (running a test match right now, seems to be working fine so far).

[Tord Romstad]

I can't reproduce exactly:

Code: Select all

 2     +0.32   00:00      200 Bxa6 c3+ Ke2 Nxb4
 2     +0.36   00:00      287 b5 c3+ Kd3
 2     +0.61   00:00      366 c3 Nc7
 3     +0.65   00:00      459 c3 Nc7 Be4
 3     +0.85   00:00      688 Bxd5 exd5 b5 axb5 axb5
 4     +0.85   00:00      878 Bxd5 exd5 b5 axb5 axb5
 5     +0.69   00:00     2066 Bxd5 exd5 c3 Kd6 b5 axb5 axb5
 6     +0.77   00:00     3778 Bxd5 exd5 b5 axb5 axb5 Kd6 Ke3
 7     +0.48   00:00     6517 Bxd5 exd5 b5 axb5 axb5 Kd6 Ke3 Kc5
 7     +0.57   00:00     9575 c3 Nc7 Ke3 e5 Bc6 Ne6 b5
 8     +0.53   00:00    12635 c3 Nc7 Ke3 e5 Bc6 Kd6 b5 Ne6
 9     +0.81   00:00    25930 c3 Nc7 b5 axb5 a5 Kd6 a6
10     +1.58   00:00    37647 c3 Nc7 b5 axb5 a5 Kd6 a6 e5 a7 f5 Ke3 e4
11     +1.17   00:00    57485 c3 Nc7 b5 axb5 a5 Kd6 a6 Kc5 a7 b4 cxb4+ Kxb4 g4
                              Kc5 a8=Q Nxa8 Bxa8
12     +1.41   00:00   106141 c3 Nc7 b5 axb5 a5 Kd6 a6 Kc5 a7 b4 cxb4+ Kxb4 Ke3
                              Kc5 Ke4
13     +1.17   00:00   174577 c3 Nc7 b5 axb5 a5 Kd6 a6 Kc5 a7 b4 cxb4+ Kxb4 g4
                              Kc5 a8=Q Nxa8 Bxa8
14     +1.58   00:00   797988 c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 b4 cxb4 Nb5 Be4
                              Na7 g3
15     +1.62   00:00   998777 c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 b4 cxb4 Nb5 Be4
                              Na7 g3 c3 Kd3
16     +1.66   00:01    1321k c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 b4 cxb4 Nb5 Be4
                              h5 Bf3 g6 Be4
17     +1.45   00:01    1686k c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 b4 cxb4 Nb5 Be4
                              g6 g3 Na7
18     +1.62   00:02    3023k c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 b4 cxb4 Nb5 Be4
                              g6 g4 Na7 Kd2 Nb5 Ke3 Na7
19     +1.49   00:03    4840k c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 Kc5 a7 b4 cxb4+
                              Kxb4 Ke4 c3 Kd3 Kc5 a8=Q Nxa8 Bxa8 Kb4 Bd5 f5
20     +1.37   00:06    9351k c3 Nc7 Ke3 e5 b5 axb5 a5 Kd6 a6 Kc5 a7 b4 cxb4+
                              Kxb4 Ke4 c3 Kd3 Kb3 Be4 Kb2 Kc4 c2 Bxc2 Kxc2 Kc5
20     +1.62   00:16   22492k Bxd5 exd5 Ke3 h5 Kd4 Kd6 g4 hxg4 hxg4 f6 f4 Kc6
                              f5 Kd6 c3 Kc6 a5 Kd6 b5 axb5 a6
21     +1.54   00:23   32267k Bxd5 exd5 Ke3 h5 Kd4 Kd6 g4 hxg4 hxg4 f6 f4 Kc6
                              g5 fxg5 fxg5 Kd6 b5 axb5 axb5 g6 c3 Ke6
22     +1.82   00:33   46492k Bxd5 Kf6 Bxc4
23     +1.82   00:49   68404k Bxd5 exd5 Ke3 h5 Kd4 Kd6 g4 hxg4 hxg4 f6 f4 Kc6
                              g5 fxg5 fxg5 Kd6 g6 Kc6 c3 Kd6 a5 Kc6 Ke5 d4 Kxd4
24     +2.02   01:26  118352k Bxd5 exd5 Ke3 h5 Kd4 Kd6 g4 h4 g5 Kc6 f3 Kd6 f4
                              Kc6 Ke5 c3 f5 f6+ gxf6 gxf6+ Kd4 Kd6 b5 a5 Kxc3
                              Kc5 Kd3

However, the PV at iteration 22 is similarly weird. I'll try to figure out what happens there. By the way, looking at the search output above reminds me that we should indicate fail highs and fail lows in the search log. The search log code is very old, and was written before aspiration windows were added.

[zamar]

I wonder if it would be a good idea to disable aspiration windows in analysis mode, to make sure the user always get exact scores and high-quality PVs.

It depends for what purpose people use the engine:

1) For those who want to take a quick glance at each position of the game, it is definetily useful to disable aspiration window. (These people often use MultiPV-mode where aspiration window is automatically disabled)

2) For those who make long analysis (correspondence chess players) or those making opening analysis or otherwise investigating deeply one position, aspiration window is a good thing, because it speeds up search.

3) those running test cases how well engine can find best move in specific set of positions. Aspiration window is a must here.

So I think that forcing people to use/not to use aspiration window in analysis mode is a bad thing. Should we make this ucioption?