Formalizing the Universal Chess Interface

Discussion of chess software programming and technical issues.

Moderator: Ras

Fulvio
Posts: 396
Joined: Fri Aug 12, 2016 8:43 pm

Re: Formalizing the Universal Chess Interface

Post by Fulvio »

hgm wrote: Wed Jan 04, 2023 10:22 am As long as you make it clear that it is version 2 of the protocol, and not version 2 of the specification. Perhaps you could start the document with a sentence like:

"Those who do not mind that their engine might not work on every UCI GUI can built it to the following specifications."
That's a little bit too harsh.
He clearly put into it some effort. And a document that also includes recommendations drawn from common practice may be very useful. Like the one already mentioned: beware that some engines may ignore go infinite. Or beware that they may decide to ignore common options, like scorpio where MultiPV is not a spin and do something strange that I do not remember.
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Formalizing the Universal Chess Interface

Post by hgm »

Well, if it is true, it is not really relevant how much work was put into it. I put a lot of effort in extending CECP. That still doesn't make CECP "UCI version 2".
User avatar
Ras
Posts: 2696
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Formalizing the Universal Chess Interface

Post by Ras »

expositor wrote: Wed Jan 04, 2023 1:59 amIt's already the case that engines need to choose to send `score cp` in order for e.g. autoresignation to work with a particular GUI; nothing requires engines to send this field.
It's not about what you could theoretically do "on the line" - nobody cares. Over and over again: it's the ecosystem. It's not hard, but as long as you refuse to consider the ecosystem or don't even see what that means, failure of this UCIv2 is probably the best outcome of this endeavour.
hgm wrote: Wed Jan 04, 2023 10:17 amAnd how much good is that going to do you? You would still have to wait out the entire game for a bestmove which would never come.
This is so irrelevant in practice that CECP doesn't even prescribe that use case as mandatory.
A proper implementation of 'isready' should check whether the search is still running. E.g. by having the search store the time it last checked the clock, and comparing whether that is not too long before the moment the 'isready' was received.
Yeah, or a watchdog-like thing where upon receiving isready, the input thread sets some watchdog variable and replies once search has reset it, which it could do every so many nodes.
And what is so hacky about 'PeekNamedPipe'?
It's Windows-only, and using an input thread that is allowed to block on stdin is much simpler.
Rasmus Althoff
https://www.ct800.net
expositor
Posts: 60
Joined: Sat Dec 11, 2021 5:03 am
Full name: expositor

Re: Formalizing the Universal Chess Interface

Post by expositor »

It's not about what you could theoretically do "on the line" - nobody cares. Over and over again: it's the ecosystem. It's not hard, but as long as you refuse to consider the ecosystem or don't even see what that means, failure of this UCIv2 is probably the best outcome of this endeavour.
I believe I understand you; I think we're just talking past each other. I'm not trying to challenge the primacy of an ecosystem: a healthy ecosystem is absolutely necessary, because the actual experience using the software is the only thing that ultimately matters. My point is that the UCI ecosystem is already fragmented.

There is variation in clients' and engines' interpretations of the informal specification, variation in strictness, and wide variation in feature support (go nodes, multipv, ...). For example, some clients require the pv to be at the end of info, some don't. Some clients require multipv to always be present when the pv is present, some don't. Some clients send "go depth 245" when they want an infinite search,* some send "go infinite".

Clients and engines currently have different (incompatible) assumptions and requirements, so things are already broken. Codifying a particular behavior doesn't change that either way. (And it can never prevent people from making additions or modifications, or writing half-implementations.) However, it might make it easier for people to converge.

Admittedly, if you restrict the "UCI ecosystem" to popular engines and clients that are 5 or more years old, it is probably less fragmented than what I observe working with brand new clients and new engines. I'm motivated by the observation that new engine developers often have to make fixes or write hacks whenever they want to use a new client.

The indictment could be made, "they simply misunderstood the specification" or "they were lazy" or "they wrote an incorrect implementation", and those are all likely true! I'm interested in why that happens and what can be done to reduce the chance of it.

What seems most useful for new engine and client developers is a set of tests that would tell you exactly what your engine (or client) does correctly and what it gets wrong. That first requires deciding exactly what may or may not be assumed by engines and clients. (So my ruckus about "protocol version 2" is mostly just a prelude to what I hope will be the beneficial thing.)

I lean toward asking more from clients in order to simplify engine implementation because there are fewer clients than engines. I'm planning to reach out to client developers and ask them if they'd be willing to make their clients more careful or lenient (if needed), and then I want to help them in whatever way I can.

At least, that's the long-term vision. I expect a project like this to take several years. And maybe, like you say, it would be better if it failed.


*This happens to be Stockfish's maximum depth, so I assume that's where the number came from.
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Formalizing the Universal Chess Interface

Post by hgm »

Whatever the specifications are, there will always be developers that violate those. The more precise and restrictive the specifications are, the more often they will be violated. The longer and more elaborate they are, the more people will not bother reading them, and consequently violate them.

What would help is publish a maximally simple example of a correct implementation. Then all the lazy people that would not bother carefully reading specs would clone that.

And it would probably be a good idea to develop a 'malicious' client that subjects an engine to all kinds of nasty boundary cases, and detects in what aspects they fail.

None of this seems to require a change in the protocol.
syzygy
Posts: 5695
Joined: Tue Feb 28, 2012 11:56 pm

Re: Formalizing the Universal Chess Interface

Post by syzygy »

expositor wrote: Wed Jan 04, 2023 1:59 am
This does not mean that one could not later define an UCIv2 that is mostly compatible with UCI and which will only be used between engines and GUIs which agree on it. (E.g. the engine could list a "UCI_v2" option, and the GUI could enable UCIv2 by triggering UCI_v2.)
This is more or less what I'm proposing (hence the `protocol` remark that the draft introduces).
I do think that "Formalizing the Universal Chess Interface", i.e. removing as much as possible the unclear and ambiguous parts of the specification while being ruthless in keeping compatibility with the major GUIs and as many engines as possible, could be a useful exercise. But clearly that is not what you are planning to do.

By UCI_v2 I don't mean an "improved" protocol specification but an extension of the protocol to introduce features missing from UCI (so UCI itself wouldn't have to be hacked and incompatibilities introduced). Instead of an UCI_v2 option it could also be UCI_<feature> as Fulvio suggests. Basically how Chess960 is done in UCI: if the engine's "uci" command does not list UCI_Chess960, the GUI will not do anything with it, and if the GUI does not set UCI_Chess960, the engine will not do anything with it.

If you want to define a new, "better" protocol, I would suggest you skip UCI and do something with XML (ideally some subset for which a lightweight and efficient library exist that is available on all platforms, so programmer's don't have to reinvent the wheel). Then you get full Unicode suport for free. The UCI approach of mixing tokens separated by whitespace and parameter names and values which are allowed to include whitespace is too hopeless imho (and inventing an escape-character mechanism will just lead to headaches). Chances of adoption are pretty slim, but perhaps you can think of a feature not supported by UCI that turns out to be something everyone wants.
expositor
Posts: 60
Joined: Sat Dec 11, 2021 5:03 am
Full name: expositor

Re: Formalizing the Universal Chess Interface

Post by expositor »

I appreciate the discussion so far – it's been really useful and has changed my mind about a few things. It's also been somewhat disappointing, though, since by "feedback [on the draft]" I was asking for specific critiques.

For example, things like "in 5.6, it says that `lowerbound` and `upperbound` have to come after `cp int `, and all engines seem to do that, but clients should actually be required to handle both orderings because...".

(Your first comment in this thread, hgm, is also a good example of what I was hoping for!)
Last edited by expositor on Fri Jan 06, 2023 3:39 am, edited 1 time in total.
syzygy
Posts: 5695
Joined: Tue Feb 28, 2012 11:56 pm

Re: Formalizing the Universal Chess Interface

Post by syzygy »

expositor wrote: Fri Jan 06, 2023 3:24 am For example, things like "in 5.6, it says that `lowerbound` and `upperbound` have to come after `cp int `, and all engines seem to do that, but clients should actually be required to handle both orderings because...".
If you do want to "complete" the spec of the UCI protocol and not produce a new protocol (that should then definitely not be called UCI), then I would say there is no fixed order except probably for the pv being at the end.

Give maximum freedom to the engine programmer as long as the major GUIs don´t assume otherwise.
expositor
Posts: 60
Joined: Sat Dec 11, 2021 5:03 am
Full name: expositor

Re: Formalizing the Universal Chess Interface

Post by expositor »

I would say there is no fixed order except probably for the pv being at the end.
Will do! I'll include that in the next draft.
thomasahle
Posts: 94
Joined: Thu Feb 27, 2014 8:19 pm

Re: Formalizing the Universal Chess Interface

Post by thomasahle »

expositor wrote: Fri Jan 06, 2023 3:24 am For example, things like "in 5.6, it says that `lowerbound` and `upperbound` have to come after `cp int `, and all engines seem to do that, but clients should actually be required to handle both orderings because...".
I actually recently implemented this with `cp lowerbound int`, I think, or maybe just `lowerbound cp int`, I'm not sure. But it didn't work in any clients. I didn't understand why, since I thought I had followed the UCI protocol. So I thought it was just clients not supporting upper/lower bounds. Only when I saw how stockfish did it, and copied that, did it suddenly work.

I'm definitely a big fan of a more formal uci spec!