Weird stockfish behaviour at low depths

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
mvk
Posts: 589
Joined: Tue Jun 04, 2013 8:15 pm

Weird stockfish behaviour at low depths

Post by mvk » Sun Nov 29, 2015 1:40 pm

I noticed something interesting.

[D] 4KN2/8/3k4/8/8/6r1/8/8 b - -
The position is a draw, but look at this:

Code: Select all

./stockfish-6-sse42 
 ... snip ... 
position fen 4KN2/8/3k4/8/8/6r1/8/8 b - - 0 1 
go 
info depth 1 seldepth 1 multipv 1 score cp 29 nodes 26 nps 26000 tbhits 0 time 1 pv g3g1 
info depth 2 seldepth 2 multipv 1 score cp 4434 nodes 53 nps 26500 tbhits 0 time 2 pv g3g1 f8e6 d6e6 
info depth 3 seldepth 4 multipv 1 score cp 4434 nodes 214 nps 107000 tbhits 0 time 2 pv g3g7 f8d7 g7d7 
info depth 4 seldepth 4 multipv 1 score cp 4434 nodes 257 nps 128500 tbhits 0 time 2 pv g3g7 e8d8 g7g1 f8g6 g1g6 
info depth 5 seldepth 5 multipv 1 score cp 4434 nodes 293 nps 146500 tbhits 0 time 2 pv g3g7 e8d8 g7g1 f8g6 g1g6 
info depth 6 seldepth 6 multipv 1 score cp 34 nodes 1318 nps 439333 tbhits 0 time 3 pv g3g7 e8d8 g7g1 d8e8 g1c1 e8d8
Notice the not-quite-IGM-level PV and score from iteration 2 to 5. For some reason the knight must be sac'ed at low depth. Why would it do something like that? There are quite a few positions with such behaviour.
[Account deleted]

mvk
Posts: 589
Joined: Tue Jun 04, 2013 8:15 pm

Re: Weird stockfish behaviour at low depths

Post by mvk » Sun Nov 29, 2015 1:53 pm

Even weirder, entering an infinite loop:

[D] 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1

Code: Select all

position fen 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1
go depth 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013920 nps 3668860 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013922 nps 3668861 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013924 nps 3668862 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013926 nps 3668862 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013928 nps 3668863 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013930 nps 3668864 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013932 nps 3668864 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013934 nps 3668865 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
 ... ad infinitum ...
(To my surprise, this position comes from a CCRL game, not from Dann...)
[Account deleted]

User avatar
velmarin
Posts: 1600
Joined: Mon Feb 21, 2011 8:48 am

Re: Weird stockfish behaviour at low depths

Post by velmarin » Sun Nov 29, 2015 2:10 pm

mvk wrote:
(To my surprise, this position comes from a CCRL game, not from Dann...)
Very interesting, you can provide the game. :o

syzygy
Posts: 4457
Joined: Tue Feb 28, 2012 10:56 pm

Re: Weird stockfish behaviour at low depths

Post by syzygy » Sun Nov 29, 2015 5:51 pm

mvk wrote:I noticed something interesting.

[D] 4KN2/8/3k4/8/8/6r1/8/8 b - -
I think this is the known and accepted result of allowing move pruning in the PV.

SF looking a bit silly at low depths

The 2nd position seems to expose an undeniable bug.

mvk
Posts: 589
Joined: Tue Jun 04, 2013 8:15 pm

Re: Weird stockfish behaviour at low depths

Post by mvk » Sun Nov 29, 2015 10:36 pm

I was trying to extract the evaluation residual, but I can use a different method, so no real worries.

I now understand the first. Suggestion is then to suppress printing of such bogus plies, which might even help at the picosecond timecontrols stockfish is being tuned at. Or better, only do this pruning a bit further away from root.

Nr 2 does not happen with the uci `go' parameters normally passed when playing games. Besides, the position is lost already, so even it it were triggered then, it is not detectable in the fishtest statistics. The game would be adjudicated before anyway. So I suppose "triple-no" to any possibility of a fix.
[Account deleted]

User avatar
lantonov
Posts: 216
Joined: Sun Apr 13, 2014 3:19 pm

Re: Weird stockfish behaviour at low depths

Post by lantonov » Mon Dec 07, 2015 9:47 am

The latest SF (051215) shows normal eval at depth 3

Code: Select all

position fen 4KN2/8/3k4/8/8/6r1/8/8 b - -
go
info depth 1 seldepth 1 multipv 1 score cp 29 nodes 26 nps 26000 tbhits 0 time 1 pv g3g1
info depth 2 seldepth 2 multipv 1 score cp 4434 nodes 112 nps 56000 tbhits 0 time 2 pv g3g7 f8d7
info depth 3 seldepth 3 multipv 1 score cp 34 nodes 171 nps 85500 tbhits 0 time 2 pv g3g7 e8d8 g7g1
info depth 4 seldepth 4 multipv 1 score cp 4434 nodes 208 nps 69333 tbhits 0 time 3 pv g3g7 e8d8 g7g1 f8g6 g1g6
info depth 5 seldepth 5 multipv 1 score cp 4434 nodes 243 nps 81000 tbhits 0 time 3 pv g3g7 e8d8 g7g1 f8g6 g1g6
info depth 6 seldepth 7 multipv 1 score cp 34 nodes 1137 nps 284250 tbhits 0 time 4 pv g3g7 e8d8 g7a7 d8e8 a7e7 e8d8 e7g7
[/quote]

User avatar
lantonov
Posts: 216
Joined: Sun Apr 13, 2014 3:19 pm

Re: Weird stockfish behaviour at low depths

Post by lantonov » Mon Dec 07, 2015 9:54 am

mvk wrote:Even weirder, entering an infinite loop:

[D] 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1

Code: Select all

position fen 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1
go depth 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013920 nps 3668860 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013922 nps 3668861 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013924 nps 3668862 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013926 nps 3668862 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 upperbound nodes 11013928 nps 3668863 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013930 nps 3668864 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013932 nps 3668864 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
info depth 1 seldepth 1 multipv 1 score mate 5 lowerbound nodes 11013934 nps 3668865 time 3002 pv h8g8
info depth 1 currmove h8g8 currmovenumber 1
 ... ad infinitum ...
(To my surprise, this position comes from a CCRL game, not from Dann...)
For this position the latest SF (051215) does not give infinite eval:

Code: Select all

position fen 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1
go
info nodes 24582 time 14
bestmove h8g8
[/code]

mvk
Posts: 589
Joined: Tue Jun 04, 2013 8:15 pm

Re: Weird stockfish behaviour at low depths

Post by mvk » Mon Dec 28, 2015 8:22 am

lantonov wrote:
mvk wrote:Even weirder, entering an infinite loop:

Code: Select all

position fen 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1
go depth 1
(To my surprise, this position comes from a CCRL game, not from Dann...)
For this position the latest SF (051215) does not give infinite eval:

Code: Select all

position fen 7k/QQQQR3/2B5/4KN1Q/3QQ3/8/8/4R3 b - - 0 1
go
info nodes 24582 time 14
bestmove h8g8
You typed a different command. The infinite loop is still present in stockfish 7 beta 1. http://www.talkchess.com/forum/viewtopi ... 732#653732
[Account deleted]

Post Reply