a question about Stockfish Cowardness

[tpetzke]

No offense intended. I realize that this is the general forum and users like to chitchat.

I just finished another 15.000 games to test a patch in my engine and I'm still not convinced that it is good since the improvement is still within the error bar.

For a programmer 73 games just mean nothing, so don't be disappointed if Marco doesn't comment on your little experiment.

[GenoM]

Run another 15 000, hope it helps to decide

[Paul Bedrey]

Here is the result for Stockfish 222 coward. 45% against Houdini tactical is not too shabby! That's 35 elo less than tactical and this is 4cpu too.


Engine Score Ho
1: Houdini_3_x64 [Tactical] 55.0/100 ····································································································
2: Stockfish-222-sse42-ja (coward) 45.0/100 =0==1010010=0==1=11===11==01=0111=0=00====1=010001==00=1==0101000=10010===0001=10=====01=1=0=0==0010

100 games played / Tournament is finished
Name of the tournament: Arena tournament
Site/ Country: PAULBEDREY-PC, United States
Level: Blitz 4/2
Hardware: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz with 4,096 MB Memory
Operating system: Windows 7 Home Premium Home Edition (Build 7600)
PGN-File: C:\Program Files (x86)\Arena\Tournaments\Coward.pgn
Website:
E-Mail Address:

[Dr.Wael Deeb]

Run another 15 000, hope it helps to decide

Ti napravo go razbi totalno momcheto

[Paul Bedrey]

Here are the disapointing results for Stockfish 4. At one point Stockfish was at 63% but seemed to run out of gas and settled for a 51.5% win. This seems to be within what Stockfish 4 is currently scoring against Houdini so it does not appear to be a big improvement.


Engine Score St
1: Stockfish_13082619_x64_modern_sse42 51.5/100 ····································································································
2: Houdini_3_x64 48.5/100 000====0=====1====10=00==0==00====01====01=01=0=01=====1=100===1=10=1=011=1==1==1===0===0=01101=1===

100 games played / Tournament is finished
Name of the tournament: Arena Mini tournament
Site/ Country: PAULBEDREY-PC, United States
Level: Blitz 4/2
Hardware: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz with 4,096 MB Memory
Operating system: Windows 7 Home Premium Home Edition (Build 7600)
PGN-File: C:\Program Files (x86)\Arena\Tournaments\coward.pgn
Website:
E-Mail Address:

[GenoM]

Run another 15 000, hope it helps to decide

Ti napravo go razbi totalno momcheto

Preuvelichavat znachenieto na statistikata... 15 000 bili malko...

[Dr.Wael Deeb]

I'll take a closer look when I have more time........
Dr.D

[Dr.Wael Deeb]

Run another 15 000, hope it helps to decide

Ti napravo go razbi totalno momcheto

Preuvelichavat znachenieto na statistikata... 15 000 bili malko...

[GenoM]

The number of the games will always be too small to conclude anything So I'm looking at the games sometime. And sometime I can see some things. Like that SF (incl. SF4) sometime makes unjustified terrible mistakes. Such as that one:

52... f3 (??!)

[pgn][Event "Stockfish_130819_gnt"]
[Site "SMARTKID"]
[Date "2013.08.25"]
[Round "10"]
[White "Sting SF 2"]
[Black "Stockfish 130819"]
[Result "1-0"]
[ECO "A07"]
[Time "15:09:04"]
[TimeControl "5+0"]
[Termination "adjudication"]
[PlyCount "111"]

1. g3 d5 2. Nf3 Nf6 3. Bg2 c6 4. O-O Bg4 5. d3 Nbd7 6. Nbd2 e5 7. h3 Bh5 8.
c4 Bd6 9. g4 Bg6 10. cxd5 cxd5 11. Nh4 Nb6 12. a4 Qe7 13. a5 Nbd7 14. a6
bxa6 15. g5 Nh5 16. Nc4 Nf4 17. Bxf4 exf4 18. Bxd5 Qxg5+ 19. Bg2 Bc7 20.
Nf3 Qe7 21. Nfe5 Rc8 22. Nc6 Qg5 23. Nxa7 Rb8 24. Nc6 Bf5 25. e3 Rb5 26.
Qf3 O-O 27. e4 Be6 28. Rxa6 Nb8 29. Nxb8 Rfxb8 30. Rc1 Bd8 31. Ra4 Rb3 32.
Ra3 Bxc4 33. Rxc4 Rxb2 34. Rac3 g6 35. Rc8 Kg7 36. d4 R8b4 37. d5 Bb6 38.
Rc2 R4b3 39. Qg4 Qe5 40. Bf3 h5 41. Qg2 Qd4 42. Kh2 Rxc2 43. Rxc2 Qd3 44.
Rc6 Qxf3 45. Qxf3 Rxf3 46. Rxb6 Rxf2+ 47. Kg1 Re2 48. Rb4 Kf6 49. Rd4 Ke7
50. Kf1 Re3 51. h4 Kd6 52. Kf2 f3 53. Kxe3 Ke7 54. e5 f2 55. Rf4 Ke8 56.
Rxf2 {Arena Adjudication} 1-0 [/pgn]

that's a just one example, but I witnessed more of a such weird mistakes. If I have to I'll try to find them. I want to ask more experienced testers and developers: is it normal behavior for SF?

regards,
Geno

[tmokonen]

It's not just you, Geno. I have noticed such behavior when using a faster time control with no increment, such as game in 1 minute, when there is little time left on the clock (say, 3 seconds or less left at 1'). It will be a drawish endgame, then all of a sudden SF throws away a piece for nothing. I have not tried out SF4 yet, but I did notice this with SF3 and many of the development versions on the Abrok website.