Stockfish time management is BADLY broken

[Jouni]

Here is another time loss played with default settings happens also around time control:

[pgn]
[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "Houdini 6 x64-pext"]
[Black "Stockfish 220917 64 BMI2"]
[Result "1-0"]
[SetUp "1"]
[FEN "rnbqk2r/1ppp1ppp/4pn2/p7/1bPP4/P4N2/1P1BPPPP/RN1QKB1R b KQkq - 0 1"]
[PlyCount "118"]

1... Bxd2+ 2. Qxd2 O-O 3. Nc3 d5 4. e3 b6 5. cxd5 exd5 6. Bd3 Bb7 7. Qc2 Nbd7
8. O-O Re8 9. Rac1 c6 10. Rfe1 g6 11. h3 Qc7 12. a4 Qd6 13. Ne2 Rac8 14. b3 Ne4
15. Nf4 h6 16. Bxe4 Rxe4 17. Qd3 Qf6 18. Rc3 Ree8 19. Rec1 Qd6 20. h4 Rc7 21.
R3c2 Nf6 22. Ne5 c5 23. Nf3 Ne4 24. Qe2 Qf6 25. Qb5 Rce7 26. Ne2 Kg7 27. Nc3
Ra8 28. Qf1 Ree8 29. dxc5 Nxc5 30. Nb5 Red8 31. Rc3 Ba6 32. Qd1 Rac8 33. Nbd4
Kh7 34. R1c2 Re8 35. g3 Kg7 36. h5 Kg8 37. hxg6 fxg6 38. Ne1 Rc7 39. Qf3 Kg7
40. Qg4 Rf7 41. Qh4 Qd6 42. Rc1 Qe5 43. Ng2 Kh7 44. Qh2 Qd6 45. f3 g5 46. g4
Qf6 47. Qh5 Rff8 48. Ne1 Bc8 49. Nec2 Bd7 50. Qh2 Qg6 51. Rd1 Rf7 52. Ne1 Qf6
53. Nec2 Kg8 54. b4 Nxa4 55. Rb3 h5 56. gxh5 g4 57. f4 Rc8 58. Ra1 g3 59. Qh1
Nc3 60. bxa5 1-0

[/pgn]

With this time control SF loses to Houdini (won all increment matches)

Code: Select all

1.  Stockfish 220917 64 BMI2 	59.0/126	20-28-78  	(L: m=0 t=2 i=0 a=26)	(D: r=51 i=13 f=1 s=0 a=13)	(tpm=1031.7 d=24.29 nps=2186427)
2.  Houdini 6 x64-pext       	67.0/126	28-20-78  	(L: m=0 t=0 i=0 a=20)	(D: r=51 i=13 f=1 s=0 a=13)	(tpm=1096.6 d=20.28 nps=2440699)

[syzygy]

So we ask once more: what program are you using to run these matches?

[Adam Hair]

From the format of latest results he posted, it appears that Jouni is using LittleBlitzer to run the matches.

[Eelco de Groot]

I get time losses from Stockfish 180917 on a repeating time control. Ridiculous that basic time management should be broken.

Sorry to hear that. Maybe to clarify a little what happened, the previous time management I think was largely Joona's work but he seems to have very little time. I don't think he has followed Fishtest at least I have not seen him comment. Gary is also extremely busy it seems. But I have seen him once or twice in this forum in the past year. So it comes down much to Marco and if he can simplify he would rather do that unless it can be shown it is a bad idea.

So any information that is detailed enough to make any conclusion I think Marco Costalba would welcome that. Other work done on time control have not yet led to much improvement and I must say I often don't know what they are talking about. I am not willing to do a lot of effort trying to understand what is written which is probably my bad.

All changes in Stockfish are open for technical comments, it only takes a registration for anyone who has a real technical interest in Stockfish development.

[Damir]

Here is another time loss played with default settings happens also around time control:

[pgn]
[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "Houdini 6 x64-pext"]
[Black "Stockfish 220917 64 BMI2"]
[Result "1-0"]
[SetUp "1"]
[FEN "rnbqk2r/1ppp1ppp/4pn2/p7/1bPP4/P4N2/1P1BPPPP/RN1QKB1R b KQkq - 0 1"]
[PlyCount "118"]

1... Bxd2+ 2. Qxd2 O-O 3. Nc3 d5 4. e3 b6 5. cxd5 exd5 6. Bd3 Bb7 7. Qc2 Nbd7
8. O-O Re8 9. Rac1 c6 10. Rfe1 g6 11. h3 Qc7 12. a4 Qd6 13. Ne2 Rac8 14. b3 Ne4
15. Nf4 h6 16. Bxe4 Rxe4 17. Qd3 Qf6 18. Rc3 Ree8 19. Rec1 Qd6 20. h4 Rc7 21.
R3c2 Nf6 22. Ne5 c5 23. Nf3 Ne4 24. Qe2 Qf6 25. Qb5 Rce7 26. Ne2 Kg7 27. Nc3
Ra8 28. Qf1 Ree8 29. dxc5 Nxc5 30. Nb5 Red8 31. Rc3 Ba6 32. Qd1 Rac8 33. Nbd4
Kh7 34. R1c2 Re8 35. g3 Kg7 36. h5 Kg8 37. hxg6 fxg6 38. Ne1 Rc7 39. Qf3 Kg7
40. Qg4 Rf7 41. Qh4 Qd6 42. Rc1 Qe5 43. Ng2 Kh7 44. Qh2 Qd6 45. f3 g5 46. g4
Qf6 47. Qh5 Rff8 48. Ne1 Bc8 49. Nec2 Bd7 50. Qh2 Qg6 51. Rd1 Rf7 52. Ne1 Qf6
53. Nec2 Kg8 54. b4 Nxa4 55. Rb3 h5 56. gxh5 g4 57. f4 Rc8 58. Ra1 g3 59. Qh1
Nc3 60. bxa5 1-0

[/pgn]

With this time control SF loses to Houdini (won all increment matches)

Code: Select all

1.  Stockfish 220917 64 BMI2 	59.0/126	20-28-78  	(L: m=0 t=2 i=0 a=26)	(D: r=51 i=13 f=1 s=0 a=13)	(tpm=1031.7 d=24.29 nps=2186427)
2.  Houdini 6 x64-pext       	67.0/126	28-20-78  	(L: m=0 t=0 i=0 a=20)	(D: r=51 i=13 f=1 s=0 a=13)	(tpm=1096.6 d=20.28 nps=2440699)

Introduce Slow Mover parameter that has been deleted, and there won't be any time losses...

[Guenther]

From the format of latest results he posted, it appears that Jouni is using LittleBlitzer to run the matches.

The problem is that we need much more infos for a real bug report.
I also don't see if it is e.g. ponder on/multi cpu.
Also the games seem to be started from a FEN and I don't know what LittleBlitzer sends regarding move count and so on...

OTH it seems the user is not really interested in finding out what happened.
(the given pgn is obviously not what originally was saved as game file too)

I started a longer test with 4 SF versions at mps (between 07-10/2017), but currently only with single cpus and ponder off to get at least concurrency=2 w/o making my machine unusable for other things.

First test used 40/30 and tm=10 (first sanity check => tm=10 is very small, but here there are zero time losses for all participants)
Second will use 60/30 and tm=0

Results probably at the weekend.

[Jouni]

Actually partial reason for time losses can be just LittleBlitzer! It starts game always with move 1. even if there is 10 move pgn opening - no good. It hides time management after book - use more time for first moves etc. I think LittleBlitzer saves real PGN? Fritz GUI with move animation has no time losses. LittleBlitzer is not good for real test at all? Sorry for false subject.

[Guenther]

Actually partial reason for time losses can be just LittleBlitzer! It starts game always with move 1. even if there is 10 move pgn opening - no good. It hides time management after book - use more time for first moves etc. I think LittleBlitzer saves real PGN? Fritz GUI with move animation has no time losses. LittleBlitzer is not good for real test at all?

There is some missleading info on the LittleBlitzer site.
You cannot use all cpus simultaneously in a meaningful test, even if
you don't do anything on that machine and close resident processes and
virus scanner and whatever. You should at least leave one cpu for
other OS tasks.
This should be general knowledge of testers though.

From the homepage of LittleBlitzer

Code: Select all

Some of the major features:

    Can allocate per-engine time with sub-millisecond accuracy.
    Very low tournament overheads so you can allocate every available CPU to the chess engines. Utilises LittleThought's efficient move generation code to maximise performance.
    Can run multiple engine vs engine games in parallel - for example, on a 4-core machine you can run 4 single-cpu matches, or on a 8-core machine you can run 4 dual-cpu matches.

...

[Guenther]

Actually partial reason for time losses can be just LittleBlitzer! It starts game always with move 1. even if there is 10 move pgn opening - no good. It hides time management after book - use more time for first moves etc. I think LittleBlitzer saves real PGN? Fritz GUI with move animation has no time losses. LittleBlitzer is not good for real test at all? Sorry for false subject.

http://talkchess.com/forum/viewtopic.ph ... tleblitzer

About a possible wrong move counter (it seems LB translates PGN to a FEN,
but still starts from move 1).
This could also explain your problem.

[syzygy]

Actually partial reason for time losses can be just LittleBlitzer! It starts game always with move 1. even if there is 10 move pgn opening - no good. It hides time management after book - use more time for first moves etc. I think LittleBlitzer saves real PGN? Fritz GUI with move animation has no time losses. LittleBlitzer is not good for real test at all? Sorry for false subject.

Unless LittleBlitzer really feeds the engine misleading time information, which seems unlikely, there is nothing wrong with the subject line of this thread. Stockfish should easily be able to play 60 moves per minute without getting flagged.

Looking at the code, I suspect this is not a simple off-by-one error but too aggressive time allocation.