Martin on the SF loss on time

[Modern Times]

I must admit I was surprised to read how Martin decidedly rejected the idea of just increasing this lag parameter. It seemed the natural thing to do (and in fact not setting it to a safer value was a mistake in the first place)..

Indeed. In the past, Martin has changed Komodo's default contempt parameter on request from the Komodo team hasn't he ? I thnk that is the case, but I could be mistaken. Perhaps he has changed the rules sicne then.

[syzygy]

I must admit I was surprised to read how Martin decidedly rejected the idea of just increasing this lag parameter. It seemed the natural thing to do (and in fact not setting it to a safer value was a mistake in the first place)..

Indeed. In the past, Martin has changed Komodo's default contempt parameter on request from the Komodo team hasn't he ? I thnk that is the case, but I could be mistaken. Perhaps he has changed the rules sicne then.

You are mistaken. And not setting the lag parameter to a safer value was indeed a mistake, but not one that Martin made.

[hgm]

And not setting the lag parameter to a safer value was indeed a mistake, but not one that Martin made.

Sure, it was a mistake of the Stockfish team. But it doesn't seem one worth spilling blood over, and adjusting it to a more realistic value seems a lot less drastic action then changing to a completely different version.

I don't know how much testing time they got on the actual TCEC hardware before the stage started, but if it wasn't very much it seems a bit harsh to say "you overestimated the speed of my machine a little, and I am going to make you pay deerly for that". If during the testing on the TCEC machine no problems occurred, then Stockfish must have been extremely unlucky to encounter it twice now, and there really wouldn't be much reason to change anything at all, as it would be unlikely to happen again.

[Psyck]

There is probably a couple of days of testing before the season and an even shorter period of testing between stages. These blitz tests are probably meant to be more of a sanity check as you cannot expect them to be tested at the time controls that the games are actually run at.

Not finding an issue in these tests obviously does not guarantee anything as there have been several crashes and other defects showing up during that games that were not detected in testing.

[hgm]

Well, if participants are allowed to replace their engine between stages, the replacements deserve to be tested as thoroughly as the initialentries. Note that testing at ast TC would be expected to reveal bugs like using too much time for stopping, if they were done at a TC where you are always close to being flagged. (Like 10 sec + 1 sec/move.)

[Norm Pollock]

SF team put in a substantially different version for Stage 3 than they did for Stage 2.

It probably had more "housekeeping" chores with regard to running the new SMP on 20 cores. Closing down 20 cores each using tbs (in endgame situations where the time losses occurred) after each move probably used more housekeeping than the previous version.

If that is the case, they did not anticipate the difference, or give the engine enough error slack. If they left the time management the same, they were negligent.

[syzygy]

And not setting the lag parameter to a safer value was indeed a mistake, but not one that Martin made.

Sure, it was a mistake of the Stockfish team. But it doesn't seem one worth spilling blood over, and adjusting it to a more realistic value seems a lot less drastic action then changing to a completely different version.

I agree that it would have been better not to make an exception just because it's SF. Either consider this a play limiting bug and let the SF team provide a fix as per the rules (i.e. increase the lag parameter), or let SF continue with the current version and parameter settings until the end of the stage.

Actually, I wonder why reverting to the previous version should eliminate the problem if the lag parameter is not increased at the same time. Maybe there is a technical reason for this, but the mere fact that this problem occurred only in the current stage and twice at that could just be a matter of good luck in the previous stages and of bad luck in the current stage.

I don't know how much testing time they got on the actual TCEC hardware before the stage started, but if it wasn't very much it seems a bit harsh to say "you overestimated the speed of my machine a little, and I am going to make you pay deerly for that".

It seems more a case of underestimating the time lost in stopping threads (and/or suboptimally handling this). At any rate it should have been clear that 10ms is nothing on an OS that still counts time in 1/60ths of a second. (But I would not rely on such a tiny safety margin on Linux either.)

[BubbaTough]

I must admit I was surprised to read how Martin decidedly rejected the idea of just increasing this lag parameter. It seemed the natural thing to do (and in fact not setting it to a safer value was a mistake in the first place)..

Indeed. In the past, Martin has changed Komodo's default contempt parameter on request from the Komodo team hasn't he ? I thnk that is the case, but I could be mistaken. Perhaps he has changed the rules sicne then.

My understanding is that the lag parameter was broken and being ignored, so changing the parameter would not help.

-Sam

[syzygy]

My understanding is that the lag parameter was broken and being ignored, so changing the parameter would not help.

-Sam

Who told you?

Strictly speaking we didn't get the chance to fix any bug although that was possible simply changing an UCI parameter default value (it is a time management parameter): we have asked TCEC to set a different UCI parameter value before the game, but this was not accepted, and I think tournament managers have for sure a point in not accepting it...but then unilateral reverting to a version _they_ choose is does not seem to me a natural consequence.

It seems more likely that reverting to the previous version will not really fix the problem.

If the participants voted based on misinformation, that seems unfortunate. (However, I might not be aware of all the details of this case myself.)

[Graham Banks]

TCEC now seems to be a commercial enterprise, so it was no real surprise that they'd find a way to ensure that there would be the expected Stockfish v Komodo final.