UCI Engine Tuning

[Michel]

make the engine stateless.

A similar issues occurs with the hash tables. Engines guess if the position belongs to a game or not in order to keep the previous information (tables, killers etc.), but that is not truly stateless, and the protocol was modified to acknowledge this, with a kludge.

Miguel. This has been discussed before. You are confusing the protocol state with the engine state. The UCI _protocol_ is mostly stateless, but engines can keep whatever state they find convenient.

[Zach Wegner]

If you look at the XML vs JSON debate you will get a taste of what I'm saying. XML is more comprehensive and does more, but people are moving away from XML and towards JSON in droves because it's conceptually simpler and more elegant.

I think this is the most important point. UCI isn't that great, but the xboard protocol is so huge, complicated, and confusing that UCI is clearly superior (yes, I changed my mind. I used to be a big xboard advocate).

Just a few examples of really ugly things in the xboard protocol (all IMO of course):

• A mess of obsolete, useless, or redundant commands (white/black, random, playother, remove, edit, hint, name, rating, computer, pause, resume). Even if new engine authors don't have to implement these, they still have to read a lot of protocol specification about them, and of course remember to tell xboard not to send those commands with some feature command.
• So many different features were added at different points, with different engines adding only some of these, that an entire system for negotiating what commands can be used in the protocol is required. I don't know of any engine that even implements the "rejected" command, because it's easier to just refuse to work than try and work around weird GUIs. A large amount of the protocol specification describes obsolete features, and there's even an "Idioms and backwards compatibility" section. The amount of the protocol specification dedicated to describing strange behaviors and workarounds for obsolete buggy engines is huge.
• Being stateful for the current position is one thing (it saves a lot of communication overhead, for one), but things like which side the engine is playing and time control should be stateless. The "force" command should be completely unnecessary, and analyze mode too.
• A very weird option syntax (*option1 /// option2???). It unnecessarily overloads the feature command, and uses silly Windows-API based names like "spin". It offloads save/restore state capabilities to the engine (which is just a very bad design decision), and through this even allows engines a way to send another set of "feature" commands, presumably with only "option" features.
• While resigning and drawing are possible, they are poorly defined. The protocol tries to suggest the best times to send them. Resigning/offering a draw should be done atomically with a move (in UCI a natural extension to support this would be "bestmove e2e4 resign"), and their effect should be well defined.
• Illegal moves and positions are fully legal in the protocol, and the engine is responsible for rejecting them. How this is arbitrated when an engine rejects another engine's move is left fully unspecified. Illegal positions can't even be rejected in any meaningful way. The RESULT command is another bad result of this. It would be much better if the protocol assumed that the interface is the arbiter of move/position legality.
• Thinking output is basically unspecified. Mate scores, score perspective, and PVs are things that are bare-minimum requirements for a useful protocol.
• Lots of commands have ridiculous names that exist only because the protocol is based on the command line interface of a twenty year old program (easy/hard, post/nopost).
• ICS commands baked into the protocol
• The xboard protocol is still evolving. New features are being added that, while maybe useful, most people don't even bother to implement.

[michiguel]

make the engine stateless.

A similar issues occurs with the hash tables. Engines guess if the position belongs to a game or not in order to keep the previous information (tables, killers etc.), but that is not truly stateless, and the protocol was modified to acknowledge this, with a kludge.

Miguel. This has been discussed before. You are confusing the protocol state with the engine state. The UCI _protocol_ is mostly stateless, but engines can keep whatever state they find convenient.

But that is part of my point. It would be a better if they match.

The UCI GUI looks like a control freak but sometimes gives freedom with incomplete information.

I think that the GUI should handle all given aspects of an issue or none of them. For instance with time management (correct me if I'm wrong), the GUI knows what time control follows the current one. However, it send info about the current one only. That is a problem, because it leaves the final decision of what to do to the engine, which does not have all the information. I think it would be better to let the engine know everything and make a decision, or the GUI make a decision and let the engine know only what is the suggested time (and panic) time for this move.

Miguel

[michiguel]

If you look at the XML vs JSON debate you will get a taste of what I'm saying. XML is more comprehensive and does more, but people are moving away from XML and towards JSON in droves because it's conceptually simpler and more elegant.

I think this is the most important point. UCI isn't that great, but the xboard protocol is so huge, complicated, and confusing that UCI is clearly superior (yes, I changed my mind. I used to be a big xboard advocate).

One of the biggest flaws of the xb protocol is the documentation, which is one of the strengths of the UCI.

I understand the philosophy behind supporting older engines and GNUchess, but you do not have to keep it in the forefront of the documentation! you can have a link to an appendix at the end decribing WB I etc. etc.

This is not a criticism ot HGM at all, since this problem predates him.

Miguel

Just a few examples of really ugly things in the xboard protocol (all IMO of course):

• A mess of obsolete, useless, or redundant commands (white/black, random, playother, remove, edit, hint, name, rating, computer, pause, resume). Even if new engine authors don't have to implement these, they still have to read a lot of protocol specification about them, and of course remember to tell xboard not to send those commands with some feature command.
• So many different features were added at different points, with different engines adding only some of these, that an entire system for negotiating what commands can be used in the protocol is required. I don't know of any engine that even implements the "rejected" command, because it's easier to just refuse to work than try and work around weird GUIs. A large amount of the protocol specification describes obsolete features, and there's even an "Idioms and backwards compatibility" section. The amount of the protocol specification dedicated to describing strange behaviors and workarounds for obsolete buggy engines is huge.
• Being stateful for the current position is one thing (it saves a lot of communication overhead, for one), but things like which side the engine is playing and time control should be stateless. The "force" command should be completely unnecessary, and analyze mode too.
• A very weird option syntax (*option1 /// option2???). It unnecessarily overloads the feature command, and uses silly Windows-API based names like "spin". It offloads save/restore state capabilities to the engine (which is just a very bad design decision), and through this even allows engines a way to send another set of "feature" commands, presumably with only "option" features.
• While resigning and drawing are possible, they are poorly defined. The protocol tries to suggest the best times to send them. Resigning/offering a draw should be done atomically with a move (in UCI a natural extension to support this would be "bestmove e2e4 resign"), and their effect should be well defined.
• Illegal moves and positions are fully legal in the protocol, and the engine is responsible for rejecting them. How this is arbitrated when an engine rejects another engine's move is left fully unspecified. Illegal positions can't even be rejected in any meaningful way. The RESULT command is another bad result of this. It would be much better if the protocol assumed that the interface is the arbiter of move/position legality.
• Thinking output is basically unspecified. Mate scores, score perspective, and PVs are things that are bare-minimum requirements for a useful protocol.
• Lots of commands have ridiculous names that exist only because the protocol is based on the command line interface of a twenty year old program (easy/hard, post/nopost).
• ICS commands baked into the protocol
• The xboard protocol is still evolving. New features are being added that, while maybe useful, most people don't even bother to implement.

[Don]

Both protocols follow a different philosophy. In both there are "technical" issues (like resolving string or file names etc.) and design issues. Winboard is more faithful to its philosophical design than UCI. UCI was supposed to be stateless, despite some of its fans denied it in previous discussions, but has some hybrid things that make it a bit clumsy. For instance, time management. It is is truly stateless, it should receive a command that says "Panic time is 2 minutes, do not ever exceed that, suggested time: 10 seconds". In that way, the GUI will make all the decisions regarding time management and make the engine stateless.

A similar issues occurs with the hash tables. Engines guess if the position belongs to a game or not in order to keep the previous information (tables, killers etc.), but that is not truly stateless, and the protocol was modified to acknowledge this, with a kludge.

I don't see your point. The protocol provides a position to the engine and issues a go command and then engine searches it. That is the abstraction the protocol provides and what internal state the engine might keep is none of the protocols business.

It doesn't take a rocket scientist however to see that the engine can easily determine if this is part of the same game if it choose to manipulate it's own state but the protocol need not force some assumption on it. It's the protocol that is stateless, not the engine.

There is a command that I think was added to UCI that you might be referring to which is called ucinewgame. I'm not sure what the purpose of that was supposed to be but I think it was a conceptual blunder to add it and it's redundant with "position startpos" - if it has a different purpose it should have been renamed accordingly.

UCI is very well suited for a engine used as an analysis tool, but winboard's design fits better an actual game.

Yes, UCI may sound easier, but that is because it relieves responsibilities to the GUI.

Bingo!

It doesn't just sound easier, it IS easier. It's designed for engine authors, not GUI designers - there are about 100 times more engines than GUI's.

At one point I thought I could separate roles into two executables 1) a polyglot-like adaptor who handles lots of things including different protocols, books, TBs, and 2) a uci-like engine. That is, basically, splitting my current engine in two. But if I ever do that, I wouldn't use UCI and I would write my own. It would look very similar to UCI, but it should be truly stateless, and if it remembers any part of a state, it should be explicitly told what to remember.

Miguel

Ideally you want to have clear separation of responsibilities and minimal interaction. Black box as much as you can. UCI appeals to me because it does relegate responsibility AWAY from me the programmer for a lot of things in order to simplify my job. I would not like that if it had any kind of noticeable impact on the strength of the program but I don't believe that it does.

What I see as a virtue someone else see's as a curse I suppose.

[Don]

If you look at the XML vs JSON debate you will get a taste of what I'm saying. XML is more comprehensive and does more, but people are moving away from XML and towards JSON in droves because it's conceptually simpler and more elegant.

I think this is the most important point. UCI isn't that great, but the xboard protocol is so huge, complicated, and confusing that UCI is clearly superior (yes, I changed my mind. I used to be a big xboard advocate).

One of the biggest flaws of the xb protocol is the documentation, which is one of the strengths of the UCI.

I understand the philosophy behind supporting older engines and GNUchess, but you do not have to keep it in the forefront of the documentation! you can have a link to an appendix at the end decribing WB I etc. etc.

This is not a criticism ot HGM at all, since this problem predates him.

At the very least HGM should define a "modern" subset of the protocol and deprecate everything else and whatever documentation there is for the deprecated stuff should be thrown out or put in an appendix at the end.

Miguel

Just a few examples of really ugly things in the xboard protocol (all IMO of course):

• A mess of obsolete, useless, or redundant commands (white/black, random, playother, remove, edit, hint, name, rating, computer, pause, resume). Even if new engine authors don't have to implement these, they still have to read a lot of protocol specification about them, and of course remember to tell xboard not to send those commands with some feature command.
• So many different features were added at different points, with different engines adding only some of these, that an entire system for negotiating what commands can be used in the protocol is required. I don't know of any engine that even implements the "rejected" command, because it's easier to just refuse to work than try and work around weird GUIs. A large amount of the protocol specification describes obsolete features, and there's even an "Idioms and backwards compatibility" section. The amount of the protocol specification dedicated to describing strange behaviors and workarounds for obsolete buggy engines is huge.
• Being stateful for the current position is one thing (it saves a lot of communication overhead, for one), but things like which side the engine is playing and time control should be stateless. The "force" command should be completely unnecessary, and analyze mode too.
• A very weird option syntax (*option1 /// option2???). It unnecessarily overloads the feature command, and uses silly Windows-API based names like "spin". It offloads save/restore state capabilities to the engine (which is just a very bad design decision), and through this even allows engines a way to send another set of "feature" commands, presumably with only "option" features.
• While resigning and drawing are possible, they are poorly defined. The protocol tries to suggest the best times to send them. Resigning/offering a draw should be done atomically with a move (in UCI a natural extension to support this would be "bestmove e2e4 resign"), and their effect should be well defined.
• Illegal moves and positions are fully legal in the protocol, and the engine is responsible for rejecting them. How this is arbitrated when an engine rejects another engine's move is left fully unspecified. Illegal positions can't even be rejected in any meaningful way. The RESULT command is another bad result of this. It would be much better if the protocol assumed that the interface is the arbiter of move/position legality.
• Thinking output is basically unspecified. Mate scores, score perspective, and PVs are things that are bare-minimum requirements for a useful protocol.
• Lots of commands have ridiculous names that exist only because the protocol is based on the command line interface of a twenty year old program (easy/hard, post/nopost).
• ICS commands baked into the protocol
• The xboard protocol is still evolving. New features are being added that, while maybe useful, most people don't even bother to implement.

[Zach Wegner]

I don't see your point. The protocol provides a position to the engine and issues a go command and then engine searches it. That is the abstraction the protocol provides and what internal state the engine might keep is none of the protocols business.

It doesn't take a rocket scientist however to see that the engine can easily determine if this is part of the same game if it choose to manipulate it's own state but the protocol need not force some assumption on it. It's the protocol that is stateless, not the engine.

There is a command that I think was added to UCI that you might be referring to which is called ucinewgame. I'm not sure what the purpose of that was supposed to be but I think it was a conceptual blunder to add it and it's redundant with "position startpos" - if it has a different purpose it should have been renamed accordingly.

ucinewgame is actually quite useful. The protocol actually assumes you keep the state of hash tables, killers, etc. between each move. "ucinewgame" is for an actual new game (which you don't get from position startpos), and all the state should be cleared.

Bingo!

It doesn't just sound easier, it IS easier. It's designed for engine authors, not GUI designers - there are about 100 times more engines than GUI's

And yet, UCI is also easier to implement for an interface, too.

[michiguel]

Both protocols follow a different philosophy. In both there are "technical" issues (like resolving string or file names etc.) and design issues. Winboard is more faithful to its philosophical design than UCI. UCI was supposed to be stateless, despite some of its fans denied it in previous discussions, but has some hybrid things that make it a bit clumsy. For instance, time management. It is is truly stateless, it should receive a command that says "Panic time is 2 minutes, do not ever exceed that, suggested time: 10 seconds". In that way, the GUI will make all the decisions regarding time management and make the engine stateless.

A similar issues occurs with the hash tables. Engines guess if the position belongs to a game or not in order to keep the previous information (tables, killers etc.), but that is not truly stateless, and the protocol was modified to acknowledge this, with a kludge.

I don't see your point. The protocol provides a position to the engine and issues a go command and then engine searches it. That is the abstraction the protocol provides and what internal state the engine might keep is none of the protocols business.

It doesn't take a rocket scientist however to see that the engine can easily determine if this is part of the same game if it choose to manipulate it's own state but the protocol need not force some assumption on it. It's the protocol that is stateless, not the engine.

There is a command that I think was added to UCI that you might be referring to which is called ucinewgame. I'm not sure what the purpose of that was supposed to be but I think it was a conceptual blunder to add it and it's redundant with "position startpos" - if it has a different purpose it should have been renamed accordingly.

UCI is very well suited for a engine used as an analysis tool, but winboard's design fits better an actual game.

Yes, UCI may sound easier, but that is because it relieves responsibilities to the GUI.

Bingo!

It doesn't just sound easier, it IS easier. It's designed for engine authors, not GUI designers - there are about 100 times more engines than GUI's.

Obviously, but it is easier not because a task became easier, but because a task that that you had now you don't have. If I want to duplicate the roles I have in my engine, I cannot do it with UCI. I have to implement my "polyglot" + my "engine". My "polyglot" will resign the way I want, do learning the way I want, and examine book holes the way I want. My engine is both, xboard, and UCI, but in UCI mode, I cannot easily do any of those things. So it is easier, until I realize I need to write my own polyglot or GUI to duplicate my xboard capabilities.

Miguel

At one point I thought I could separate roles into two executables 1) a polyglot-like adaptor who handles lots of things including different protocols, books, TBs, and 2) a uci-like engine. That is, basically, splitting my current engine in two. But if I ever do that, I wouldn't use UCI and I would write my own. It would look very similar to UCI, but it should be truly stateless, and if it remembers any part of a state, it should be explicitly told what to remember.

Miguel

Ideally you want to have clear separation of responsibilities and minimal interaction. Black box as much as you can. UCI appeals to me because it does relegate responsibility AWAY from me the programmer for a lot of things in order to simplify my job. I would not like that if it had any kind of noticeable impact on the strength of the program but I don't believe that it does.

What I see as a virtue someone else see's as a curse I suppose.

No, that is not the point because I can clearly see the advantages. I meant to say that UCI fell short in some aspects to have a "clear separation of responsibilities" as you say. Time management is one.

Miguel

[michiguel]

I don't see your point. The protocol provides a position to the engine and issues a go command and then engine searches it. That is the abstraction the protocol provides and what internal state the engine might keep is none of the protocols business.

It doesn't take a rocket scientist however to see that the engine can easily determine if this is part of the same game if it choose to manipulate it's own state but the protocol need not force some assumption on it. It's the protocol that is stateless, not the engine.

There is a command that I think was added to UCI that you might be referring to which is called ucinewgame. I'm not sure what the purpose of that was supposed to be but I think it was a conceptual blunder to add it and it's redundant with "position startpos" - if it has a different purpose it should have been renamed accordingly.

ucinewgame is actually quite useful. The protocol actually assumes you keep the state of hash tables, killers, etc. between each move. "ucinewgame" is for an actual new game (which you don't get from position startpos), and all the state should be cleared.

That is exactly the one I meant. It should have been in the first version.

Miguel

Bingo!

It doesn't just sound easier, it IS easier. It's designed for engine authors, not GUI designers - there are about 100 times more engines than GUI's

And yet, UCI is also easier to implement for an interface, too.

[stevenaaus]

I think this discussion is completely misdirected.

The poll is about _options_ in UCI versus xboard. Not about the general merits of the two protocols.

The poll also asks a secondary question: how should options be used for _engine tuning_?.

The correct answer to the primary question is of course "neutral". There is very little difference in the handling of options between xboard and UCI. Unfortunately "neutral" was not provided as a possible answer.

The second question has nothing to do with the primary question.

Since it is impossible to answer two questions in one poll, I propose that
the poll be closed and replaced by a properly worded one.

Yes, you're right Michel. The poll and question don't bear much resemblence, so sorry for that. I just added the poll as an after thought.