UCI protocol: giving up!

[tpetzke]

Hm, I don't think UCI will lead to high traffic by itself. If you see a logfile spammed it is usually because an engine outputs current move info, but no one forces an engine to do that.

I have current move info as an option that defaults to false. The info strings sent during search are not so much traffic usually and contain useful stuff in a standardized format.

Probably it makes sense to filter them if a many of them are sent in a short time frame (like in fine 70 or so), I haven't done that yet but have it on my list.

Thomas...

[mar]

Hm, I don't think UCI will lead to high traffic by itself. If you see a logfile spammed it is usually because an engine outputs current move info, but no one forces an engine to do that.

I have current move info as an option that defaults to false. The info strings sent during search are not so much traffic usually and contain useful stuff in a standardized format.

Probably it makes sense to filter them if a many of them are sent in a short time frame (like in fine 70 or so), I haven't done that yet but have it on my list.

Thomas...

I send currmove ~10 times a second or so because of that. There's also multipv where UCI mandates all pv lines to be sent together. Also it always repeats all moves for each position. This has an overhead beause the engine has to parse and make all the moves.

Martin

[ilari]

Hm, I don't think UCI will lead to high traffic by itself. If you see a logfile spammed it is usually because an engine outputs current move info, but no one forces an engine to do that.

It's not just about the amount of data that's being sent to the GUI. A bigger problem is the horrible syntax that makes it difficult and very CPU-intensive to parse the info strings. If only multi-word values had to be wrapped in quotes... In addition to case-insensitive option names, this is my biggest complaint about the UCI protocol. An other annoyance with UCI is engine options being used to represent engine features, which confuses users and engine developers and makes life harder for GUI developers. Xboard's "feature" strings are so much better.

Xboard/Winboard suffers from different problems:
- The protocol was originally hacked to work with GnuChess and its inconsistencies and bugs. Without the GnuChess legacy the protocol wouldn't suggest crazy things like sending a SIGINT signal to stop the engine's thinking.
- No millisecond precision and multiple different time formats. UCI gets this right by only using milliseconds.
- I don't see why the protocol should support more than 1 move format. I don't think SAN is a good format for a chess engine communication protocol.
- Some misleading command names like "hard" and "easy".
- Inconsistencies in setting options (eg. "egtpath TYPE PATH" vs. "option NAME=VALUE").
- Doesn't specify a clear format for PV output or even which player's point of view is used for the eval score.
- Many engine developers don't give a damn about following the protocol specs because Xboard, Winboard and ChessGUI will accept pretty much anything the engine throws at them.

I would conclude that both protocols are poorly designed, but we just have to live with them. And I do applaud HGM's almost herculean efforts to keep Xboard/Winboard alive. I couldn't do it myself.

[lucasart]

Hm, I don't think UCI will lead to high traffic by itself. If you see a logfile spammed it is usually because an engine outputs current move info, but no one forces an engine to do that.

It's not just about the amount of data that's being sent to the GUI. A bigger problem is the horrible syntax that makes it difficult and very CPU-intensive to parse the info strings. If only multi-word values had to be wrapped in quotes... In addition to case-insensitive option names, this is my biggest complaint about the UCI protocol. An other annoyance with UCI is engine options being used to represent engine features, which confuses users and engine developers and makes life harder for GUI developers. Xboard's "feature" strings are so much better.

Xboard/Winboard suffers from different problems:
- The protocol was originally hacked to work with GnuChess and its inconsistencies and bugs. Without the GnuChess legacy the protocol wouldn't suggest crazy things like sending a SIGINT signal to stop the engine's thinking.
- No millisecond precision and multiple different time formats. UCI gets this right by only using milliseconds.
- I don't see why the protocol should support more than 1 move format. I don't think SAN is a good format for a chess engine communication protocol.
- Some misleading command names like "hard" and "easy".
- Inconsistencies in setting options (eg. "egtpath TYPE PATH" vs. "option NAME=VALUE").
- Doesn't specify a clear format for PV output or even which player's point of view is used for the eval score.
- Many engine developers don't give a damn about following the protocol specs because Xboard, Winboard and ChessGUI will accept pretty much anything the engine throws at them.

I would conclude that both protocols are poorly designed, but we just have to live with them. And I do applaud HGM's almost herculean efforts to keep Xboard/Winboard alive. I couldn't do it myself.

Yes, that's a good summary. Both have problems, although the problems of the UCI protocol are more on the GUI than the engine side.
Really it wouldn't take much to fix the UCI protocol, like the quotes you mention, or simplify and standardize further the info lines. Also I would add that certain options should be defined by the standard (hash size should be calles Hash, nothing else, same for Ponder and Threads)

But the problem is that:
* xboard, due to its prehistoric legacy is full of a lot of things that should simply be removed (some programs send moves in SAN others in long algebraic notation, some even write things like "2. ... Nxd4+" or whatever)
* UCI is more of a de facto standard than a real standard. It is defined by Stefan MK and his document on UCI hasn't evolved for many years.

IMO a standardization comittee should be formed for both Xboard and UCI, in order to define the new xboard and uci standard in a way that is more efficient, robust, simple (for engine developpers as well as gui developpers) and as standard as possible.
And of course this comittee should be formed with GUI developpers.

[wgarvin]

Also your program will benefit from the book of the GUI. One of the many advantages of UCI vs Xboard

???

I guess you mean: one of the many lies spread spead by proponents of a flaky and inefficient protocol to trick people into using it.

Whether programs use GUI book is of course a matter of GUI implementation, and has absolutely nothing to do with what protocol is used to communicate with the engine, as in this case the GUI is playing from the book, and the engine is not even involved...

100% agree... UCI is horrible, the spec is extremely informal and loose, but more importantly: It's not your ENGINE playing chess, its the combination of "engine plus GUI". That may make sense for commercial packages that include a couple of engines plus a GUI. [Aside: what GUI did Shredder come with? I'm not much familiar with the history...]

However, it does not make sense in the current world where we have GUIs for the user, and chess engines for the chess-playing. These two concerns should be completely separated, IMO. GUIs should just display boards, accept user inputs, show information such as game history and PVs to the user, and communicate information between the two engines (or the engine and the user, etc.) Engines should be able to play a complete game from beginning to end without assistance, and should be able to do this under any reasonable GUI that talks in a protocol the engine understands.

I don't think GUIs should be making chess-playing decisions such as choosing book moves or adjudicating tablebase draws--if the purpose of a chess engine is to, you know, play chess, then the engine should be capable of playing all of the moves of the game without the GUI hand-holding it or taking over the move decisions for part of the game. If there is user demand for shared book formats, shared tablebase formats, etc. then the chess programming community should develop standalone libraries that implement a book format or a tablebase format, and common GUI options for selecting books and tablebase directories, and then encourage all of the engine authors to support these common components. But building this stuff into the GUI is not the right solution! Among other problems, it means that engines may play differently under different GUIs, because the GUI is tampering with the game.

So yeah... the Xboard/Winboard protocol may have a lot of legacy quirks, but at least the spec is somewhat rigorous, and it leaves things like books and tablebases for the engine to be in charge of.


Side note: Maybe the computer chess community should develop a library that handles BOTH the Winboard/Xboard and the UCI protocols, as robustly as possible, and provides a simple and standardized interface to the engine. [Edit: Polyglot might be a good resource.] Then new engine authors could just use this library for I/O to the GUI, and focus their efforts on the chess parts of the engine. Standardizing a simple-but-flexible API would be hard. But there would be definite benefits:
* Engines using the library would work equally well under any GUI
* The library could automatically handle "legacy" GUI issues (e.g. SAN vs coord notation, using edit instead of setboard command, etc.)
* Engine authors would not have to develop their own protocol-handling routines (which often contain bugs or incomplete implementations)
* GUI authors would have a "gold standard" to test for compatibility with. If they pass the library's tests, their GUI should work well with any engine that uses the library.

[hgm]

Xboard/Winboard suffers from different problems:
- The protocol was originally hacked to work with GnuChess and its inconsistencies and bugs. Without the GnuChess legacy the protocol wouldn't suggest crazy things like sending a SIGINT signal to stop the engine's thinking.
- No millisecond precision and multiple different time formats. UCI gets this right by only using milliseconds.
- I don't see why the protocol should support more than 1 move format. I don't think SAN is a good format for a chess engine communication protocol.
- Some misleading command names like "hard" and "easy".
- Inconsistencies in setting options (eg. "egtpath TYPE PATH" vs. "option NAME=VALUE").
- Doesn't specify a clear format for PV output or even which player's point of view is used for the eval score.
- Many engine developers don't give a damn about following the protocol specs because Xboard, Winboard and ChessGUI will accept pretty much anything the engine throws at them.

Indeed, I agree with most of that. I always considered the option to use SAN , and even to request XBoard using SAN, an abomination.

There is actually a de-facto standard for score POV (which it is not in the official specs is only because the scores at the time those specs were made were not used for anything, so no one cared much). But unfortunately there are some stubborn, but vocal authors that prefer to do things their own way.

The fact that PV is free format has some good points as well; it makes it more easy to extend the protocol in a backward-compatible way.

The silly names of some commands does not worry me so much. It is at most a minor inconvenience, and I wonder if it is worth changing.

I guess I am to blame for the inhomogeneity of option settings; I should have kept a more firm stance in favor of the way I originally wanted to do this, but I bowed to popular demand... Initially I did not plan the 'option' keyword, nor the equal sign. Just "NAME VALUE", which would have been very similar to "egtpath TYPE PATH".

Do you suggest it would be better to terminate engines on any non-compliant utterance? I would not be against this, because Fairy-Max would still run, but I am afraid that it simply means we would have no users anymore, as 80% or so of all existing WB engines could no longer be run. I could make an option for this (-strict true|false) which by default would forfeit anything other than the recommended standard (even obsolete and deprecated alternatives), and make that the default setting. Problem is there are other GUIs, and people often develop their engines under those, so I don't think it would have much impact.

As to the milliseconds: The fact that the time parameters in the level command are different from those in the time command is ugly, but I don't consider it more than an annoyance. That the time commands has centi-sec resolution is becoming close to unacceptable today, but switching to millisec only delays the inevitable, as in the future even that might not suffice. Would it be an idea to extend the protocol with a feature millisec=1 which would indicate the engine would accept the times in floating point format (but still specifying centi-sec)? The GUI could then decide how many digits to send behind the decimal point depending on how much average time per move is specified.

[hgm]

I don't think GUIs should be making chess-playing decisions such as choosing book moves or adjudicating tablebase draws--if the purpose of a chess engine is to, you know, play chess, then the engine should be capable of playing all of the moves of the game without the GUI hand-holding it or taking over the move decisions for part of the game.

What you say makes perfect sense from the viewpoint of tournaments like WCCC. But you should realize that there are people that use GUIs for other purposes than identifying the best chess-playing entity. E.g. a book author could want to try his book with an engine not written by himself, with a private book format. Or play different books against each other with the same engine(s). GUIs are also meant for him. Many low-strength engines do not come with books at all. Testers could still want to test these engines for middle-game strength, using a neutral book as an efficient means for creating starting positions to ensure game diversity. GUIs are also intended to satisfy their needs. Engine developers could want to adjudicate lengthy endgames against test opponents that are suitable in other respects, but do not support EGTB, and stubbornly stretch out dead draws for several multiples of 50 moves (if they still have Pawns to push), wasting valuable CPU time.

So there are very legitimate reasons for a GUI to support an opening book or tablebase adjudication.

I agree this invites laziness on the part of engine authors, though. None of my engines implement book, I rely entirely on WB to handle it for them. (Which is of course OK for me to do, as I programmed the WB book handling myself. ) I wonder how the WCCC rules are in this respect. It would of course be OK for Shredder to let Shredder GUI handle book, but if other participants are using Shredder GUI...

[hgm]

Position and go are different.

Those insisting that UCI is a stateless protocol could not agree with you.

But the question was actually: do you consider 'go', 'ponder', 'wtime', 'btime' all as different?

If you consider them as one, you are of course 'counting yourself rich'. All implementations of WB protocol I know also send time, otim and go or time, otim and the move always as a single sequence.

As a matter of fact, a strict UCI implementation is a lot harder, because commands like wtime, btime, winc, binc do not have a fixed meaning, as in WB protocol, but a context-dependent meaning: when they appear on the same line as 'go' then in any other context (where the specs say they should be ignored). So where in WB protocol you can simply have one big switch in a single command-decoding loop, UCI forces you to have all kind of sub-loops for decoding the sub-commands in the context of each main command.

[Evert]

There is actually a de-facto standard for score POV

Which is...?
I always report the scores used internally, which are side-to-move POV. But I'd be happy to convert them if needed.

The silly names of some commands does not worry me so much. It is at most a minor inconvenience, and I wonder if it is worth changing.

Maybe. Maybe not.
Sjaak and Jazz accept "ponder on/off" in addition to "hard/easy"; almost never run them from the command line like that though.

I guess I am to blame for the inhomogeneity of option settings; I should have kept a more firm stance in favor of the way I originally wanted to do this, but I bowed to popular demand... Initially I did not plan the 'option' keyword, nor the equal sign. Just "NAME VALUE", which would have been very similar to "egtpath TYPE PATH".

Alas, it is often how these things go...

Do you suggest it would be better to terminate engines on any non-compliant utterance? I would not be against this, because Fairy-Max would still run, but I am afraid that it simply means we would have no users anymore, as 80% or so of all existing WB engines could no longer be run. I could make an option for this (-strict true|false) which by default would forfeit anything other than the recommended standard (even obsolete and deprecated alternatives), and make that the default setting.

I would welcome such an option, but I don't think it should be the default for the first version where it is introduced. Announce that it will be the default in later versions.
Or not. "Zachte heelmeesters" and all that.

As to the milliseconds: The fact that the time parameters in the level command are different from those in the time command is ugly, but I don't consider it more than an annoyance. That the time commands has centi-sec resolution is becoming close to unacceptable today, but switching to millisec only delays the inevitable, as in the future even that might not suffice. Would it be an idea to extend the protocol with a feature millisec=1 which would indicate the engine would accept the times in floating point format (but still specifying centi-sec)? The GUI could then decide how many digits to send behind the decimal point depending on how much average time per move is specified.

Sounds good to me.

[Evert]

* xboard, due to its prehistoric legacy is full of a lot of things that should simply be removed (some programs send moves in SAN others in long algebraic notation, some even write things like "2. ... Nxd4+" or whatever)

I would welcome it if some of the commands were deprecated and moved to their own section in the manual, marked "don't use these".
It's sortof mentioned at the moment, but it's not as clear as it can be in my opinion.