a question about Stockfish Cowardness

[Paul Bedrey]

Without the coward setting Stockfish 222 gets creamed by Houdini tactical!

Next match will be Stockfish 4 vs Stockfish 222 coward. Should be interesting.


Engine Score Ho
1: Houdini_3_x64 [Tactical] 70.5/100 ····································································································
2: Stockfish-222-sse42-ja [BASE] 29.5/100 001010==1=0===1=0=0====00000=000==0100==000=0=000===01000==1000=0==01100000=000100000=0=0=011000==10

100 games played / Tournament is finished
Name of the tournament: Arena tournament
Site/ Country: PAULBEDREY-PC, United States
Level: Blitz 4/2
Hardware: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz with 4,096 MB Memory
Operating system: Windows 7 Home Premium Home Edition (Build 7600)
PGN-File: C:\Program Files (x86)\Arena\Tournaments\Coward.pgn
Website:
E-Mail Address:

[zullil]

The number of the games will always be too small to conclude anything So I'm looking at the games sometime. And sometime I can see some things. Like that SF (incl. SF4) sometime makes unjustified terrible mistakes. Such as that one:

52... f3 (??!)

[pgn][Event "Stockfish_130819_gnt"]
[Site "SMARTKID"]
[Date "2013.08.25"]
[Round "10"]
[White "Sting SF 2"]
[Black "Stockfish 130819"]
[Result "1-0"]
[ECO "A07"]
[Time "15:09:04"]
[TimeControl "5+0"]
[Termination "adjudication"]
[PlyCount "111"]

1. g3 d5 2. Nf3 Nf6 3. Bg2 c6 4. O-O Bg4 5. d3 Nbd7 6. Nbd2 e5 7. h3 Bh5 8.
c4 Bd6 9. g4 Bg6 10. cxd5 cxd5 11. Nh4 Nb6 12. a4 Qe7 13. a5 Nbd7 14. a6
bxa6 15. g5 Nh5 16. Nc4 Nf4 17. Bxf4 exf4 18. Bxd5 Qxg5+ 19. Bg2 Bc7 20.
Nf3 Qe7 21. Nfe5 Rc8 22. Nc6 Qg5 23. Nxa7 Rb8 24. Nc6 Bf5 25. e3 Rb5 26.
Qf3 O-O 27. e4 Be6 28. Rxa6 Nb8 29. Nxb8 Rfxb8 30. Rc1 Bd8 31. Ra4 Rb3 32.
Ra3 Bxc4 33. Rxc4 Rxb2 34. Rac3 g6 35. Rc8 Kg7 36. d4 R8b4 37. d5 Bb6 38.
Rc2 R4b3 39. Qg4 Qe5 40. Bf3 h5 41. Qg2 Qd4 42. Kh2 Rxc2 43. Rxc2 Qd3 44.
Rc6 Qxf3 45. Qxf3 Rxf3 46. Rxb6 Rxf2+ 47. Kg1 Re2 48. Rb4 Kf6 49. Rd4 Ke7
50. Kf1 Re3 51. h4 Kd6 52. Kf2 f3 53. Kxe3 Ke7 54. e5 f2 55. Rf4 Ke8 56.
Rxf2 {Arena Adjudication} 1-0 [/pgn]

that's a just one example, but I witnessed more of a such weird mistakes. If I have to I'll try to find them. I want to ask more experienced testers and developers: is it normal behavior for SF?

regards,
Geno

[d]8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52

Stockfish never considers playing f3. This must be a GUI bug.

Code: Select all

position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go depth 20
info depth 1 seldepth 2 score cp 250 nodes 56 nps 28000 time 2 multipv 1 pv f7f6
info depth 2 seldepth 4 score cp 286 nodes 303 nps 101000 time 3 multipv 1 pv d6e5 d4a4 e3e4 a4e4 e5e4
info depth 3 seldepth 4 score cp 286 nodes 397 nps 132333 time 3 multipv 1 pv d6e5 d4a4 e3e4 a4e4 e5e4
info depth 4 seldepth 8 score cp 286 nodes 894 nps 178800 time 5 multipv 1 pv d6e5 d4a4 e3e4 a4e4 e5e4
info depth 5 seldepth 9 score cp 256 nodes 1475 nps 210714 time 7 multipv 1 pv d6e5 d4a4 f7f6 a4a6 e3e4
info depth 6 seldepth 11 score cp 294 nodes 4840 nps 302500 time 16 multipv 1 pv f7f6 d4b4 d6e5 b4a4 g6g5 a4a6 e3e4
info depth 7 seldepth 11 score cp 294 nodes 6180 nps 309000 time 20 multipv 1 pv f7f6 d4b4 d6e5 b4a4 g6g5 a4a6 e3e4
info depth 8 seldepth 16 score cp 305 nodes 49936 nps 475580 time 105 multipv 1 pv e3h3 d4a4 h3h4 a4a8 h4h3 a8d8 d6e7 d8c8 h3e3 c8c7 e7d6
info depth 9 seldepth 22 score cp 171 nodes 101665 nps 516065 time 197 multipv 1 pv e3h3 d4d1 f7f6 d1g1 d6e5 g1g6 h3d3 f2e2 e5e4 g6f6 d3d5 f6e6 e4f5
info depth 10 seldepth 24 score cp 169 nodes 137453 nps 524629 time 262 multipv 1 pv e3h3 d4d1 f7f6 d1g1 d6e5 g1g6 h3h4 g6g7 e5e4 d5d6 h4h2 f2e1 h2h1 e1d2 h1h3 d6d7 h3d3 d2e2 h5h4 g7e7
info depth 11 seldepth 30 score cp 202 nodes 156826 nps 529817 time 296 multipv 1 pv e3h3 d4d1 f7f6 d1g1 d6e5 g1g6 h3h4 g6g7 e5e4 d5d6 h4h2 f2e1 h2h1 e1d2 f4f3 g7e7 e4f4 d6d7 f3f2 d7d8q f2f1q d8b6 h1h2 d2c3 f1c1
info depth 12 seldepth 32 score cp 204 nodes 177665 nps 553473 time 321 multipv 1 pv e3h3 d4d1 f7f6 d1g1 d6e5 g1g6 h3h4 g6g7 e5e4 d5d6 h4h2 f2e1 h2h1 e1d2 f4f3 g7e7 e4f4 d6d7 f3f2 d7d8q f2f1q d8d4 f4g5 d4e3 g5g6 e3d3 f1d3 d2d3 h1d1 d3e4 d1e1
info depth 13 seldepth 39 score cp 163 nodes 376634 nps 720141 time 523 multipv 1 pv e3h3 d4d1 f7f6 d1g1 d6e5 g1g6 h3d3 g6g8 e5e4 g8e8 e4f5 e8d8 f5e5 d8e8 e5d6 e8d8 d6e7 d8h8 d3d5 f2f3 e7e6 f3f4 d5d4 f4g3
info depth 14 seldepth 39 score cp 169 nodes 547918 nps 819010 time 669 multipv 1 pv f7f6 d4a4 d6e5 a4a6 e3e4 a6e6 e5d5 e6f6 e4a4 f6g6 d5e4 g6g5 a4a2 f2e1 e4e3 g5e5 e3f3 e5h5 a2a1 e1d2 f3e4 h5g5 f4f3 g5g4 e4e5
info depth 15 seldepth 39 score cp 169 nodes 589214 nps 845357 time 697 multipv 1 pv f7f6 d4a4 d6e5 a4a6 e3e4 a6e6 e5d5 e6f6 e4a4 f6g6 d5e4 g6g5 a4a2 f2e1 e4e3 g5e5 e3f3 e5h5 a2a1 e1d2 f3e4 h5g5 f4f3 g5g4 e4e5
info depth 16 seldepth 39 score cp 224 nodes 1245120 nps 1135022 time 1097 multipv 1 pv e3a3 d4d1 a3b3 d1d4 b3e3 d4b4 d6e5 b4b7 e3e4 b7f7 e4d4 f7e7 e5f6 e7d7 d4d3 f2e2 d3e3 e2f2 e3h3 d7d6 f6f5 d6b6 h3h4
info depth 17 seldepth 39 score cp 208 nodes 1706430 nps 1299642 time 1313 multipv 1 pv e3a3 d4d1 a3b3 d1d2 b3h3 d2d1 h3a3 d1d2 a3a1 d2b2 d6e5 b2b7 e5e4 b7f7 a1a2 f2e1 a2a6 e1f2 e4e5 f2f3 a6a3 f3e2 a3e3 e2f2 e3d3 f7g7 e5e4 g7g6 d3d2 f2e1 d2d5 g6h6 f4f3 e1f2 d5d2 f2e1 f3f2 e1f1 e4e3 h6e6 e3f3 e6f6 f3e3
info depth 18 seldepth 39 score cp 208 nodes 1788879 nps 1328046 time 1347 multipv 1 pv e3a3 d4d1 a3b3 d1d2 b3h3 d2d1 h3a3 d1d2 a3a1 d2b2 d6e5 b2b7 e5e4 b7f7 a1a2 f2e1 a2a6 e1f2 e4e5 f2f3 a6a3 f3e2 a3e3 e2f2 e3d3 f7g7 e5e4 g7g6 d3d2 f2e1 d2d5 g6h6 f4f3 e1f2 d5d2 f2e1 f3f2 e1f1 e4e3 h6e6 e3f3 e6f6 f3e3
info depth 19 seldepth 39 score cp 202 nodes 2080732 nps 1414501 time 1471 multipv 1 pv e3a3 d4d1 a3b3 d1d2 b3h3 d2d1 h3a3 d1d2 a3a1 d2d3 a1b1 d3a3 d6e5 a3a7 b1b2 f2e1 e5e4 d5d6 e4e5 a7f7 e5d6 f7f4 d6e5 f4a4
info depth 20 seldepth 39 score cp 202 nodes 2335925 nps 1498348 time 1559 multipv 1 pv e3a3 d4d1 a3b3 d1d2 b3h3 d2d1 h3a3 d1d2 a3a1 d2d3 a1b1 d3a3 d6e5 a3a7 b1b2 f2e1 e5e4 d5d6 e4e5 a7f7 e5d6 f7f4 d6e5 f4a4
info nodes 2335925 time 1559
bestmove e3a3 ponder d4d1

[zullil]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

[GenoM]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

[zullil]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

Further testing shows that SF also plays f3 in sudden death when it has very little time (like 5 msecs):

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go btime 5
info nodes 2 time 2
bestmove f4f3 ponder (none)

So I'll call it a "time-pressure blunder".

[GenoM]

I suppose that better settings of Cowardice and Mobility are set as default for SF3 & SF4 because SF3&SF4 with Mobility and Cowardice aren't showing better results (as with SF 2.2.2 in Paul case and SF 2.3 in mine case)

Code: Select all

  15 Stockfish 130805                3000   32   32   308   60%  2934   41% 
  16 Strelka 5                       2999   44   43   182   63%  2901   37% 
  17 Stockfish 2.3 mobility          2992   34   34   332   66%  2851   33% 


  20 Stockfish 130819                2989   32   32   318   58%  2936   37% 
  21 Stockfish 130719                2985   38   38   245   59%  2904   34% 

  23 Stockfish 130818                2982   32   32   318   57%  2936   40% 
  24 Stockfish 2.3 coward            2977   34   34   336   63%  2857   25% 

  26 Stockfish 130809                2973   35   35   248   54%  2939   46% 
  27 Stockfish 130803                2967   38   38   208   55%  2934   47% 
  28 Stockfish 2.3.1                 2966   46   46   155   60%  2861   43% 
  29 Stockfish 4                     2964   30   30   360   53%  2940   36% 
  30 Stockfish 3                     2962   28   28   417   55%  2915   38% 

  32 Stockfish 3 coward              2956   78   74    78   74%  2744   22% 
 
  34 Stockfish 120907                2955   44   44   181   57%  2854   39% 
  35 Stockfish 2.3                   2945   31   31   351   55%  2890   36% 
  36 Stockfish 4 mobility            2941   56   56   108   52%  2921   31% 

  40 Stockfish 130825                2932   38   38   210   47%  2948   43% 

  50 Stockfish 4 coward              2920   54   55   108   48%  2922   43% 

[GenoM]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

Further testing shows that SF also plays f3 in sudden death when it has very little time (like 5 msecs):

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go btime 5
info nodes 2 time 2
bestmove f4f3 ponder (none)

So I'll call it a "time-pressure blunder".

Yes, it is "time-pressure blunder", in this very game TC is ultra-bullet 3s+0.250s; but nevertheless the engine doesn't have to make such a blunders if other engines at the same TC didn't made such.

[Lyudmil Tsvetkov]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

Further testing shows that SF also plays f3 in sudden death when it has very little time (like 5 msecs):

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go btime 5
info nodes 2 time 2
bestmove f4f3 ponder (none)

So I'll call it a "time-pressure blunder".

But maybe, Evgeni after all deserves a bit more attention

[Eelco de Groot]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

Further testing shows that SF also plays f3 in sudden death when it has very little time (like 5 msecs):

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go btime 5
info nodes 2 time 2
bestmove f4f3 ponder (none)

So I'll call it a "time-pressure blunder".

But maybe, Evgeni after all deserves a bit more attention

[d]8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52 am f3

Yes, of course this is a good observation from Evgenii. Tony Mokononen already confirmed it. I have seen some cases especially when I was still testing in the ERT tool with 40 moves in 20 seconds. More cases of timetrouble then. But I have no great ideas how to solve it, I am just glad that Louis Zulli has been able to reproduce it. The easiest explanation is that all searches of six plies or shorter (the tactical search if you like) have holes in them, because they are very heavily pruned. Still this was about the worst possible move that one of the Stockfishes could play in the position, even in timetrouble. Playing the rook to a3, or Rh3 threatening Rxh4 (from the top of my head) still seems fine and probably winning. The move is even a passed pawn push, so it is classified as 'dangerous' by Stockfish. At the root node which is a PV node, that would make no difference (no futility pruning) but one or two plies earlier, a dangerous move would also not get into the futility pruning code block. If Stockfish really had not enough time even for some 7 ply search, I suppose anything could happen and I would classify it as either bad time management or a difference between internal or GUI time. So maybe it is sometimes better to default on time then rather than produce such a blunder... But I guess that would lower elo!

Eelco

[gladius]

Actually, SF plays f3 if told to search using 0 msecs:

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go movetime 0
info nodes 2 time 2
bestmove f4f3 ponder (none)

Would a GUI ever send that to an engine?

Strange behavior from GUI side against Stockfish only

Further testing shows that SF also plays f3 in sudden death when it has very little time (like 5 msecs):

Code: Select all

Stockfish sig-4728533 64 SSE4.2 by Tord Romstad, Marco Costalba and Joona Kiiski
position fen 8/5p2/3k2p1/3P3p/3RPp1P/4r3/5K2/8 b - - 2 52
isready
readyok
go btime 5
info nodes 2 time 2
bestmove f4f3 ponder (none)

So I'll call it a "time-pressure blunder".

Thanks for tracking this down. It is indeed a time pressure blunder. If SF has <= 5ms to play a move, it plays instantly, before even searching. So it just plays a random move.