Proposal for Linux engine standard

[hgm]

We'd have to add ~/games/engines to the list. And for a 200 user system...

This is impractical, to say the least.

Not really. I don't know enough about installing stuff on Linux to know if users can make private installs of standard packages when they lack sudo power. I guess they could always do it from source. If they can, and there is a standard for what to use as $(datadir), perhaps dependent on the --prefix they gave with their "make install", it would make sense if a GUI hunting for engine profiles would also look in the current user's engine directory.

It would never look in other user's engine directories. So that there are 200 users is not a problem.

[Michel]

If we used inifile syntax, I think it would make more sense to invert it, like
Code:
[cmdargs]
xboard=...
scid=...
icsdrone=...

Because I can't imagine anything else that would be dependent on the GUI than the cmdargs, and it seems silly to have all these sections with only a single member.

One would still need to think how alien engines would be declared.
I think this is really a protocol extension. So perhaps

protocol = wb3

?

[Michel]

I think this is really a protocol extension.

Or maybe not. But the question is still how to declare an alien engine.
I don't like

Variant = alien

since "alien" has no meaning by itself as a chess variant (it is winboard slang).

So if we declare Nebiyu as

[engine]
Command=Nebiyu
Family=chess
Variant=crazyhouse
<some options to select variant crazyhouse>

then a non-alien capable GUI will happily start Nebiyu only to discover afterwards that it does not know about variant "alien"....

BTW: I don't consider this an academic question. I would like to have a good way to run Nebiyu-alien in icsdrone, without using special
case hacks.

[hgm]

Well, I tend to think of it as just a natural extension of WB protocol. Or, in other words, WB2 is just an incomplete implementation of the full WB protocol. In particular, there are new variants that are not in th eold protocol, and the protocol extensions would only be active in those variants. But new variants are added all the time to WB2 as well.

GUIs can decide which variants they support in combination with the protocol, and ignore what they do not support. SCID would probably ignore WB engines that only play gothic or crazyhouse. Just like Shredder would ignore UCI engines that ony play xiangqi. Standard XBoard would ignore engines that play a Family/Variant it does not recognize, while the Alien Edition would trust that games from a non-Chess Family can be mapped onto one of its catch-all variants (like multi or alien), and trust the xboard cmdargs to say how to do it.

I am not sure that variant alien would be difficult to implement in icsDrone. Because the latter just mediates between engine and a fully rule-aware ICS, the protocol extension that engines send board updats after every sent or received move can be handled by simply ignoring those updates. The only problem would be to handle multi-moves.

But, come to think of it, I don't think there are any ICS that support multi-move variants. If Nebiyu needs to play Crazyhouse as variant alien, with an engine option to select Crazyhouse as 'alien representative', it would just be a matter of sending it that option setting at startup, and put 'variant alien' directly after new in the initString. And refuse any games that are not crazyhouse.

Not sure if it would be possible to make it switch variant based on what the ICS requests. But in a sense this is Nebiyu's fault; playing variants that have a well-defined WB name as a catchall is asking for trouble. Catchalls are really intended for where there are no standards, and it is just between engine and user. Playing an existing variant as alien, is just creating incompatibility for no reason, as the variant is apparently not dependent on any of the extra features available in variant alien. (OK, the GUI might be. But that would not have to stop the engine from also playing it according to the standard protocol.) For what an ICS plays, the ICS should set the standard.

[lucasart]

I think there is a great need for a btter interaction beteen installing GUIs on Linux and installing engines for those GUIs. Currentlly the burdon of configuring the GUI to use the engine is on the user, who would have to go on instructions provided in the engine docs. This is a very undesirable situation.

What I had in mind is that we would decide upon some standard system directory, like /usr/share/games/engines/, where engines could 'drop' a file during their install to announce their presence, and provide some info about themselves. GUIs could then scan this directory to kno hich engines are on the system, and retrieve the info on how to configure the GUI to use it.

For Chess engines it would obviously be important to kno if the engine is UCI or WB, what command is needed to start up the engine, which game (e.g. Chess variant) it plays, if it plays only one, if it has its own opening book, how it reports the score (white POV or stm POV), what its approximate Elo is, etc. And perhaps if it has special demands for specific GUIs, by recommending some GUI-specific GUI options.

For instance, something like

UCI command = fruit
Game = chess
Book = true
Score POV = stm
XBoard options = -fSAN
SCID options = whatever
...

or

WB command = maxqi
Game = xiangqi
Book = false
Score POV = stm
XBoard options = -fSAN -boardSize 49

Any suggestions?

I don't see how this would ever become a standard. You basically want to standardize not the protocol of process communication, but the config of engines defined by the (G)UI.
For example, I use cutechess-cli, and you do that directly in command line or by modifying a *.json file.
And setting up a few engines in a (G)UI is a trivial operation, that everyone does with whatever is the (G)UI of their choice. I don't think many people would want to do a "universal" setup and ask many different (G)UI to load this setup.
I think the standard will soon be cutechess on Linux when it's ready, because frankly all Linux GUI I've tried are horrible, and I still have to use Wine+Arena in the meantime (when I want a GUI otherwise cutechess-cli). So perhaps the JSON format makes more sense.

[Michel]

GUIs can decide which variants they support in combination with the protocol, and ignore what they do not support

So then you use

Family = chess
Variant = alien

As I said I don't really like this since it hides the fact that the engine is really playing another variant.

I guess the question is: what is the meaning of "Variant". It is just the argument of the "variant" command in the wb protocol (in that case it should be specified as such), or does it have a protocol independent meaning... I would very much prefer the latter.

[stegemma]

What about to use named pipes, to save information data of the engine?

I see, for sample, this article:

http://www.linuxjournal.com/article/2156

The idea could be: when engine starts, it everytime write its information data to a named pipe (the name would be defined by the standard). The interface reads from that named pipe all information. If you have multiple engine, you just have to run any of them, to fill the pipe, and then the interface can empty the pipe, reading all runned engine data. Named pipes are available on Linux, UNIX and Windows systems so this idea could be portable.

The only problem is that user should run any engine he would use, to "register" it to the interface. This could be interesting: if you run multiple interfaces and then multiple engines, you can choose what engine register to what interface...

[hgm]

I don't see how this would ever become a standard. You basically want to standardize not the protocol of process communication, but the config of engines defined by the (G)UI.

I want to standardize the installation of engines, as is customary for Linux packages, to make it such that after apt-get install they immediately work in any GUI. Now they only work from the command line, but no one plays engines from the command-line anymore. Requiring the user to take extra configuration steps in the GUI to use engine is rely a stone-age abomination. Nowadays we should hav plug&play install.

For example, I use cutechess-cli, and you do that directly in command line or by modifying a *.json file.

Well, it is upto cutechess-cli to decide if and how to use the information provided to it by the system to make life for the user easy. I never used cutechess-cli, so I have no idea what support it offers for the user to generate these *.json files. If it doesn't offer any, then it obviously has no need for the information the engines provide, and the user can edit json files to his heart's content.

But in XBoard use of engines is only easy after they are installed through the Load Engine dialog into its internal list of engines, and it would be a great pleasure to the user if it could be put there in an automated way. Most GUIs do have such a service, and would be happy if it could be made less cumbersome to the user.

And setting up a few engines in a (G)UI is a trivial operation, that everyone does with whatever is the (G)UI of their choice. I don't think many people would want to do a "universal" setup and ask many different (G)UI to load this setup.

What you consider tivial is a huge stumbling block for almost everyone I know. You seem blind to the fact that not everyone is a programmer...

I think the standard will soon be cutechess on Linux when it's ready, because frankly all Linux GUI I've tried are horrible, and I still have to use Wine+Arena in the meantime (when I want a GUI otherwise cutechess-cli). So perhaps the JSON format makes more sense.

Well, I never heard of this format. Does it support Xiangqi? Checkers? Go? USI protocol? How would you specify in JSON that an USI engine like GPS Shogi needs certain arguments when run in the Shogidooro GUI? It seems an awful lot that you are projecting your own personal taste and very narrow interests on the rest of the World... But the World is more than just you.

[Michel]

Well, I never heard of this format

JSON stands for Javascript object notation. It has the advantage
that (unlike ini files) it can encode data structures more than two levels
deep.

It is much easier to parse than xml and _very importantly_ it is rigorously specified.

http://www.json.org/

The fact that cutechess-cli already uses it could be an argument to adopt it here as well.

[hgm]

Well, at a first glance it seems a bit troublesome that many of the things we will want to store will be text strings, which themselves will contain double quotes, slashes and backslashes, (in the engine command, in filenames) for which JSON specifies an escaping regime. This would make the resulting files very cumbersome to read.

A format where everyting is taken literally upto the end of the line whenever a text string is expected would be much more suitable in that respect.