Losing on time

[Rebel]

I'm having problems with my engine, Quadrox, losing on time and I can't imagine why. Actually, I had been using Arena for my testing and I didn't have the problem. I've now switched to cutechess-cli and I'm having lots of problems...

While mine runs fine under Arena, Fritz etc. I ran into the same problem the first time I used cutechess-cli. There is a more simple way to tackle the problem, in cutechess-cli use:

Code: Select all

timemargin=<n>::
Let engines go <n> milliseconds over the time limit.

In graphical interfaces like Arena, Fritz etc. there is an automatic built-in tollerance margin.

[hgm]

So these GUIs are broken, and you can apparently configure cutechess-cli so that it becomes broken too, in order to allow your engine to remain broken, rather than fixing it.

Really smart...

Of course when you would ever play that engine in a serious tourney then, such as on an ICS, it would flag it mercilessly, and you would lose every game on time.

[hgm]

Why "undefined behavior"? "edit" and "setboard" change the current position, so why keep the previous game history? What do you think should happen to the move history on "setboard"? I clear it, of course, since I *can't* go back to the situation prior to a "setboard", so there is nothing that the moves that happened prior to "setboard" could be used for. They simply disappear by setting the ply to 0, so "undo" and "remove" have nothing to take back any longer at that point.

The old specs did not address the problem of 'undo' in a situation with not enough game history (and indeed did not even define what 'enough game history' would mean). The only way to fix that in a backward compatible way is by making the behavior explicitly undefined. Requiring a specific behavior (like ignoring the 'undo', or in case of a 'remove' with only a single move in the history undoing that move) might retro-actively break engines that do not do that, and inspire GUI authors to design GUIs that rely on the newly specified behavior. Specifying the behavior as 'undefined' effectively forbids GUI authors to ever send undo / remove in such cases.

You are talking about chess variants that do not have a2a3 as a legal move in their initial position. That is right. But then there is "setboard".

True, but 'setboard' is not all that convenient for variants where it is not possible to define ordinary FEN (because there are more than 26 piece types, or more than two players). The 'edit' command handles multi-letter piece names (as well as multiple players) in a natural and trivial way, both for the GUI and the engine. Why force GUI and engine authors to go an extra mile in implementing a complex FEN replacement when a simple 'black' command would solve it?

Actually I am not too familiar with chess variants, so may I ask you: is there any chess variant that requires to use "edit" instead of "setboard" *and* has no a2a3 in the initial position? Only in such a case I think that there would still be a need for the "black" command.

All Shogi variants larger than Chu Shogi would sort of fit that. Of course there are always work-arounds: one could define alternatives to FEN (e.g. comma-separating all pieces or stretches of empty squares, to be able to parse the multi-character piece names, or sending separate boards for each color, so the capitals can be used to indicate start of a piece name). In Dai Shogi there are 26 unpromoted piece types, so in theory you could still get by with FEN when you would accept the King can no longer be represented as K, but should be considered a promoted Elephant (+E). All these variants have the Pawns starting on 5th-11th rank, filling up the area behind them with pieces, so a2a3 certainly would be an illegal move there.

Of course the GUI could be made smart enough to scan the board for a white Pawn, and move that. But lets face it: having to send a dummy move in order to change the side to move is nothing but a hideous kludge. If it is desirable that this should be possible, any self-respecting protocol should have a separate command for that.

[Sven]

Why "undefined behavior"? "edit" and "setboard" change the current position, so why keep the previous game history? What do you think should happen to the move history on "setboard"? I clear it, of course, since I *can't* go back to the situation prior to a "setboard", so there is nothing that the moves that happened prior to "setboard" could be used for. They simply disappear by setting the ply to 0, so "undo" and "remove" have nothing to take back any longer at that point.

The old specs did not address the problem of 'undo' in a situation with not enough game history (and indeed did not even define what 'enough game history' would mean). The only way to fix that in a backward compatible way is by making the behavior explicitly undefined. Requiring a specific behavior (like ignoring the 'undo', or in case of a 'remove' with only a single move in the history undoing that move) might retro-actively break engines that do not do that, and inspire GUI authors to design GUIs that rely on the newly specified behavior. Specifying the behavior as 'undefined' effectively forbids GUI authors to ever send undo / remove in such cases.

Why does the protocol need to know about "not enough game history"? It is the engine that simply needs to do the obvious: reset game history whenever the position is changed by "setboard", "edit", "white", or "black" (EDIT: the latter two only in the rare case that they are not ignored, of course ...). My understanding is that GUIs will not send "undo" or "remove" after "setboard" or "edit" without any move being made in the meantime (maybe this is the only point where you might want to add a clarification to the specs), so only "white" and "black" are left to consider. But both only appear in the context of "go" with those special "idioms", so not in the middle of a game. Which non-existent problem would you try to solve here? A GUI sending "undo" when there is nothing to take back is clearly broken, so you don't need an "undefined behavior" declaration for that case.

You are talking about chess variants that do not have a2a3 as a legal move in their initial position. That is right. But then there is "setboard".

True, but 'setboard' is not all that convenient for variants where it is not possible to define ordinary FEN (because there are more than 26 piece types, or more than two players). The 'edit' command handles multi-letter piece names (as well as multiple players) in a natural and trivial way, both for the GUI and the engine. Why force GUI and engine authors to go an extra mile in implementing a complex FEN replacement when a simple 'black' command would solve it?

Actually I am not too familiar with chess variants, so may I ask you: is there any chess variant that requires to use "edit" instead of "setboard" *and* has no a2a3 in the initial position? Only in such a case I think that there would still be a need for the "black" command.

All Shogi variants larger than Chu Shogi would sort of fit that. Of course there are always work-arounds: one could define alternatives to FEN (e.g. comma-separating all pieces or stretches of empty squares, to be able to parse the multi-character piece names, or sending separate boards for each color, so the capitals can be used to indicate start of a piece name). In Dai Shogi there are 26 unpromoted piece types, so in theory you could still get by with FEN when you would accept the King can no longer be represented as K, but should be considered a promoted Elephant (+E). All these variants have the Pawns starting on 5th-11th rank, filling up the area behind them with pieces, so a2a3 certainly would be an illegal move there.

Of course the GUI could be made smart enough to scan the board for a white Pawn, and move that. But lets face it: having to send a dummy move in order to change the side to move is nothing but a hideous kludge. If it is desirable that this should be possible, any self-respecting protocol should have a separate command for that.

The a2a3 kludge was certainly invented mainly for standard chess, and works for a couple of variants as well. Now that there are some variants where it does not work any longer, you need some replacement for it. You even talk about variants with more than two players; "white" and "black" don't help there either. I see only two ways out:

1) Extend FEN spec for these variants so that "setboard" solves it.

2) Extend the WB protocol spec by a clear definition for which variants the GUI will not use "setboard" but "edit" instead, even if the engine sets "feature setboard=1". Make this also dependent on a new "extended edit" feature (choose a better name!) set by the engine to declare that it is aware of the new technique (EDIT: otherwise it can't play such variants correctly). Then define a new "edit" subcommand that sets the side to move, and will only be used in that "extended edit" case. Do not use "white" or "black" in this case but something new, to avoid reanimation of commands that were already declared obsolete.

If you ask me, I'd clearly prefer 1). But if you don't like it then do 2), provided you find a way to do it in a consistent manner that also conforms to the spirit of the WB protocol

Sven

[hgm]

Why does the protocol need to know about "not enough game history"? It is the engine that simply needs to do the obvious: reset game history whenever the position is changed by "setboard", "edit", "white", or "black" (EDIT: the latter two only in the rare case that they are not ignored, of course ...). My understanding is that GUIs will not send "undo" or "remove" after "setboard" or "edit" without any move being made in the meantime (maybe this is the only point where you might want to add a clarification to the specs),

Exactly. Because under the current specs your assumption isn't based on anything, and changing the specs to require that the engine 'must do the obvious' would actually invalidate it completely: with those new specs it would be completely valid for a GUI to send 'undo' when there is nothing to undo, counting on it that an engine would ignore it in that case.

so only "white" and "black" are left to consider. But both only appear in the context of "go" with those special "idioms", so not in the middle of a game. Which non-existent problem would you try to solve here?

Well, for one, I don't consider section 13 of the engine-intf.html as part of the protocol specs. That document is an awkward mix of protocol specs and description of what XBoard does, and this section explicitly states that it handles about the latter ("xboard uses these commands ..."). One problem is that this in some cases is not even true anymore, so at the very least it should be changed to "XBoard 4.2.7 uses these commands ...". Which, of course, would be effectively the same as deleting it alltogether, as it wouldn't tell anything about the current version of XBoard, which people are likely using, or at least would like their engine to be compatible with.

As far as I am concerned most of this section is pure madness; ridiculous constructs are used to replace straightforward commands, just to work around non-compliancy of some specific engine versions. IMO doing such things do more harm than good (as the OP of this thread shows), and I would strongly recommend authors of any new GUI to copy this idiocy.

A GUI sending "undo" when there is nothing to take back is clearly broken, so you don't need an "undefined behavior" declaration for that case.

In my philosophy it would only be broken if the specs say you cannot do that.

The a2a3 kludge was certainly invented mainly for standard chess, and works for a couple of variants as well. Now that there are some variants where it does not work any longer, you need some replacement for it. You even talk about variants with more than two players; "white" and "black" don't help there either.

True. But I could of course add 'red' and 'green' commands. XBoard obviously only supports games with 2 players. But the Nebiyu engine already does support games with more players, (currently by a kludge, using different piece types for the extra players), so it is not an entirely hypothetical problem.

I see only two ways out:

1) Extend FEN spec for these variants so that "setboard" solves it.

2) Extend the WB protocol spec by a clear definition for which variants the GUI will not use "setboard" but "edit" instead, even if the engine sets "feature setboard=1". Make this also dependent on a new "extended edit" feature (choose a better name!) set by the engine to declare that it is aware of the new technique (EDIT: otherwise it can't play such variants correctly). Then define a new "edit" subcommand that sets the side to move, and will only be used in that "extended edit" case. Do not use "white" or "black" in this case but something new, to avoid reanimation of commands that were already declared obsolete.

This seems to serve no purpose other than providing a lot of extra work. The commands needed to do this are already specified, and already implemented. Why should I have to redo that? The only problem with 'edit' is that it cannot convey castling and e.p. rights, but not all forms of Chess do have castling or e.p., and in particular the known variants for which FEN becomes problematic do not have it, so that 'edit' is an excellent solution for them. Provided you can send 'black' for setting up black-to-move positions. I don't see any harm in reviving old protocol once it is neded again (as long as it continue to mean exactly the same as it has always meant).

If you ask me, I'd clearly prefer 1). But if you don't like it then do 2), provided you find a way to do it in a consistent manner that also conforms to the spirit of the WB protocol

Sven

[Sven]

Why does the protocol need to know about "not enough game history"? It is the engine that simply needs to do the obvious: reset game history whenever the position is changed by "setboard", "edit", "white", or "black" (EDIT: the latter two only in the rare case that they are not ignored, of course ...). My understanding is that GUIs will not send "undo" or "remove" after "setboard" or "edit" without any move being made in the meantime (maybe this is the only point where you might want to add a clarification to the specs),

Exactly. Because under the current specs your assumption isn't based on anything, and changing the specs to require that the engine 'must do the obvious' would actually invalidate it completely: with those new specs it would be completely valid for a GUI to send 'undo' when there is nothing to undo, counting on it that an engine would ignore it in that case.

I did not intend to write that requiring that the engine must do the obvious shall be added to the specs, maybe my wording was ambiguous here. I simply meant that it were obvious that an engine that does not reset its game history on commands that change the current positions is broken somehow. But if we agree that GUIs shall not send "undo" or "remove" if there is nothing to take back then this part of the discussion is moot anyway, so there is in fact no "undefined behavior" involved in the "undo/remove" scenario.

so only "white" and "black" are left to consider. But both only appear in the context of "go" with those special "idioms", so not in the middle of a game. Which non-existent problem would you try to solve here?

Well, for one, I don't consider section 13 of the engine-intf.html as part of the protocol specs. That document is an awkward mix of protocol specs and description of what XBoard does, and this section explicitly states that it handles about the latter ("xboard uses these commands ..."). One problem is that this in some cases is not even true anymore, so at the very least it should be changed to "XBoard 4.2.7 uses these commands ...". Which, of course, would be effectively the same as deleting it alltogether, as it wouldn't tell anything about the current version of XBoard, which people are likely using, or at least would like their engine to be compatible with.

As far as I am concerned most of this section is pure madness; ridiculous constructs are used to replace straightforward commands, just to work around non-compliancy of some specific engine versions. IMO doing such things do more harm than good (as the OP of this thread shows), and I would strongly recommend authors of any new GUI to copy this idiocy.

I fully agree on the "madness" and "idiocy" here But section 13 should not be removed completely since you can't prove that the combination of WinBoard/XBoard 4.2.7 and old GNUchess will never be used again anywhere in our world. You might want to restructure the document into a "spec" part that is valid for all GUIs and a "WinBoard/XBoard" specific part that describes behavior of these specific GUIs and possibly their specific versions. You would still have to resolve some interdependencies between both, though, e.g. the description of "feature colors" refers to the XBoard specific part.

A GUI sending "undo" when there is nothing to take back is clearly broken, so you don't need an "undefined behavior" declaration for that case.

In my philosophy it would only be broken if the specs say you cannot do that.

Right, then add a clarifying sentence about it. If sending such a misplaced "undo" were allowed for a GUI then you would allow a situation where GUI and engine are out of sync about the game: GUI thinks there is something to take back but engine thinks it is at ply 0 of the game.

The a2a3 kludge was certainly invented mainly for standard chess, and works for a couple of variants as well. Now that there are some variants where it does not work any longer, you need some replacement for it. You even talk about variants with more than two players; "white" and "black" don't help there either.

True. But I could of course add 'red' and 'green' commands. XBoard obviously only supports games with 2 players. But the Nebiyu engine already does support games with more players, (currently by a kludge, using different piece types for the extra players), so it is not an entirely hypothetical problem.

Then decide whether the WB protocol shall officially support more than two players or not. If yes then take appropriate action, part of it would be how to define which player's turn it is and when. If no then keep the current state. In either case I would expect that you either keep the current definition of "white" and "black" as "obsolete", or change it into "obsolete except for the following cases: ...". Just be consistent. It should be clear that needing "white" and "black" commands is reserved for specific cases (e.g. specific variants).

I see only two ways out:

1) Extend FEN spec for these variants so that "setboard" solves it.

2) Extend the WB protocol spec by a clear definition for which variants the GUI will not use "setboard" but "edit" instead, even if the engine sets "feature setboard=1". Make this also dependent on a new "extended edit" feature (choose a better name!) set by the engine to declare that it is aware of the new technique (EDIT: otherwise it can't play such variants correctly). Then define a new "edit" subcommand that sets the side to move, and will only be used in that "extended edit" case. Do not use "white" or "black" in this case but something new, to avoid reanimation of commands that were already declared obsolete.

This seems to serve no purpose other than providing a lot of extra work. The commands needed to do this are already specified, and already implemented. Why should I have to redo that? The only problem with 'edit' is that it cannot convey castling and e.p. rights, but not all forms of Chess do have castling or e.p., and in particular the known variants for which FEN becomes problematic do not have it, so that 'edit' is an excellent solution for them. Provided you can send 'black' for setting up black-to-move positions. I don't see any harm in reviving old protocol once it is neded again (as long as it continue to mean exactly the same as it has always meant).

Do as you like, but again: if you want to revive the "black" command then this is fine as long as you change the spec into "obsolete except for the following cases: ...".

And wouldn't you agree that extending the FEN spec for those special variants would be much nicer, easier, future-proof, and more consistent with the idea of a modern WB protocol? Why would you put any effort into that old and clumsy "edit" style? A lot of modern testing technology is based on using FENs (e.g. for opening positions), and if those specific variants shall use that you would need to provide FEN support for it anyway at least on GUI level, but then you have a FEN spec, so why shouldn't engines use it as well via "setboard"?

Sven

[hgm]

I fully agree on the "madness" and "idiocy" here But section 13 should not be removed completely since you can't prove that the combination of WinBoard/XBoard 4.2.7 and old GNUchess will never be used again anywhere in our world.

But educating people about the peculiarities of GNU Chess or XBoard 4.2.7 is not the task of this document. Do you explain in the README file that accompanies your engine that things might work differently in GNU Chess? If people want to know how XBoard 4.2.7 works then they should look in the documentation that accompanied XBoard 4.2.7. Not in the CECP specs.

You might want to restructure the document into a "spec" part that is valid for all GUIs and a "WinBoard/XBoard" specific part that describes behavior of these specific GUIs and possibly their specific versions. You would still have to resolve some interdependencies between both, though, e.g. the description of "feature colors" refers to the XBoard specific part.

I don't see a reason why the specific behavior of XBoard versions should be described anywhere. The fact that it is CECP compliant should say everything. Perhaps there could be a caveat in the specs that one should not in general assume that the GUI will send you the obvious and minimal set of commands to perform a given task, but might use arbitrarily cumbersome combinations of commands that according to the specs would eventually have the same result.

IMO a GUI should never employ work-arounds for non-compliant engines, but if it does, publishing them is actually the worst thing you can do. Because it sends the wrong signal that people can count on this behavior, and thus implement commands in other ways that their specified meaning, breaking their engine on other GUIs, or other versions of the same GUI.

Right, then add a clarifying sentence about it. If sending such a misplaced "undo" were allowed for a GUI then you would allow a situation where GUI and engine are out of sync about the game: GUI thinks there is something to take back but engine thinks it is at ply 0 of the game.

Indeed, I plan to add a sentence about this. But anything that is not specified in the protocol will in fact cause undefined behavior even in compliant engines. There is no need to actualy write that something might cause undefined behavior for such behavior to happen. Dependnig on the implementation, doing things that the engine author was told not to worry about will have different effects.

Then decide whether the WB protocol shall officially support more than two players or not. If yes then take appropriate action, part of it would be how to define which player's turn it is and when. If no then keep the current state. In either case I would expect that you either keep the current definition of "white" and "black" as "obsolete", or change it into "obsolete except for the following cases: ...". Just be consistent. It should be clear that needing "white" and "black" commands is reserved for specific cases (e.g. specific variants).

Do as you like, but again: if you want to revive the "black" command then this is fine as long as you change the spec into "obsolete except for the following cases: ...".

I would be inclined to say "obsolete except in all cases where the GUI uses it". One problem is that "obsolete" in the current specs doesn't seem to mean anything. It certainly doesn't mean that the command is not to be used. If GUI authors would take section 13 seriously, they would actually see it as a strong recommendaion to use these commands.

And wouldn't you agree that extending the FEN spec for those special variants would be much nicer, easier, future-proof, and more consistent with the idea of a modern WB protocol? Why would you put any effort into that old and clumsy "edit" style? A lot of modern testing technology is based on using FENs (e.g. for opening positions), and if those specific variants shall use that you would need to provide FEN support for it anyway at least on GUI level, but then you have a FEN spec, so why shouldn't engines use it as well via "setboard"?

Yes, that would be nice. If it could be done. But I don't agree that the old 'edit' style is intrinsically clumsy. Extensions of the FEN standard could easily become much more clumsy. Pasting positions into the GUI could definitely become a problem, however. DEMN could be a solution, but it isn't very readable to humans.

[Rebel]

Nice rant HGM, but you are missing a couple of points.

1. First of all practically none of the end-users uses cutechess-cli as their interface, it's a programmer tool (mainly) to play bullet games.

2. Secondly some engines (like mine for instance) collect and store all kind of statistic stuff which causes overhead falling outside the limits of the time control. I don't want to give that up and as long as my engine doesn't lose on time under the most popular GUI's like Fritz, Arena, Chesspartner I am extremely happy with the "timemargin" option cutechess-cli offers.

If it's not win WB I suggest you add such an option for programmers satisfaction.

[hgm]

Over my dead body!

Never will I be an accomplish to such deterioration of standards. WinBoard will remain a reliable interface.

[Sven]

2. Secondly some engines (like mine for instance) collect and store all kind of statistic stuff which causes overhead falling outside the limits of the time control.

If you do some expensive stuff in the time between terminating the search and actually sending the move to the GUI then your engine may occasionally lose on time. Would it help to send the move immediately and do the expensive stuff afterwards, when it is not your engine's turn (i.e. it is either idle or pondering, where in the latter case pondering could also start a little later)?

If that "overhead" you mentioned is performed while the GUI has your engine's clock still running then the "time" command (WB) would help you out, but you would still need to add a reliable estimate of the time needed for running that overhead activity once as a safety buffer in order to avoid losing on time. May I assume that your engine handles the "time" command in WB mode?

Sven