UCI Win/Draw/Loss reporting

[Ras]

if we would get dozens of new infos

That YAGNI will hardly make it across many engines or GUIs anyway. WDL is different because NN engines like LC0 have a different way of expressing the evaluation - not in centipawns, but in terms of probability. That's a fundamentally different kind of strong chess engine that does need some amendments.

Actually, the cleanest way would be to enhance the "score" tag to allow "score wdl X Y Z" with X Y Z in permill. The sum of all must be 1000. That's easy to implement and robust to parse. If that is triggered by a UCI_WdlScore boolean (default: false), that allows for a smooth and simple integration.

Drowning the protocol in enhancements doesn't pan out that well because that ends up with one or two engines and one GUI that offer full support.

[phhnguyen]

Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.

Table
1) Data (partly command)
- A row of a table requires data as an array. Each item has a column name and value. Value could be an integer, a real number or a string.

Code: Select all

["W": 60, "D": 20, "L": 20]

- A table may have few rows - an array of arrays:

Code: Select all

[["Ord": 5, "W": 60, "D": 20, "L": 20], ["Ord": 2, "W": 80, "D": 5, "15": 20]]

2) Table of 1 row (full command)

Code: Select all

uci json {"table": ["W": 60, "D": 20, "L": 20]}

If chess GUIs receive new data of the table, they will replace old table by new one.


3) Table of multi-rows

Code: Select all

uci json {"table": [["Ord": 1, "W": 60, "D": 20, "L": 20], ["Ord": 2, "W": 40, "D": 60, "L": 30]]}

If chess GUIs receive new data, they will replace old table by new one.

4) Table of multi-rows, update on-fly

We need a special column, named "depth", "row" or "move".

Code: Select all

uci json {"table": ["row": 1, "W": 60, "D": 20, "L": 20]}

Example data for Lc0:

Code: Select all

uci json {"table": ["move": "f2f4", "self depth": 351, "N": "33 (+ 3)", "P%": 2.98, "Q": -0.07531, "D": 0.209, "U": 0.18615, "Q+U": 0.11084, "V": -0.0719] }

If chess GUIs receive new data, they will replace the corresponding row (based on values of column "row", "depth", "move") in the table by new row. Column "depth" may be sorted descently when "row" ascently.

5) Multi tables

Tables may be distinguished by their names, the data format should be a map. Each time the engine should send data of one table only (not all multi-table data at once).

Code: Select all

uci json {"table": {"name" : "WDL stats", "data": ["row": 1, "W": 60, "D": 20, "L": 20]}}

Notes:
- The data structures to display tables are quite flexible, depend on needs.
- Reserved/key words: "table", "row", "depth", "move"
- All tables will be reset when side-to-move of the board changed

Chess GUIs may display those tables separately with other info and let users decide to view or hide them (e.g., via buttons)

[Ferdy]

Perhaps simpler and more uci way of communicating is by key value pair. Example,

Code: Select all

winprob 20 drawprob 60 lossprob 20

values like 20, 60 and 20 are in percent.

I don't see the benefit of having extra tags. We don't send a pv either with pv e4e4 pv e7e5 pv g1f3. I also don't see how it is simpler, looks the opposite to me.

In this particular case extra tag is appropriate since winprob can stand on its own. If position is already winning sending winprob 99 is simpler. There is no need to send the drawprob and lossprob. Most games are drawish, sending drawprob 99 is also simpler, most users are no longer concern of winprob and lossprob.

Regarding pv it is a different case, pv line is better presented as a sequence of moves.

[Ferdy]

Perhaps simpler and more uci way of communicating is by key value pair. Example,

Code: Select all

winprob 20 drawprob 60 lossprob 20

values like 20, 60 and 20 are in percent.

I don't see the benefit of having extra tags. We don't send a pv either with pv e4e4 pv e7e5 pv g1f3. I also don't see how it is simpler, looks the opposite to me.

In this particular case extra tag is appropriate since winprob can stand on its own. If position is already winning sending winprob 99 is simpler. There is no need to send the drawprob and lossprob. Most games are drawish, sending drawprob 99 is also simpler, most users are no longer concern of winprob and lossprob.

Regarding pv it is a different case, pv line is better presented as a sequence of moves.

I may add, uci protocol encourages clear key names, example,
info depth, the word depth is fully spelled out.
info score, the word score is fully spelled out too.
info multipv and others.

wdl is not clear, is it probability? is it a counter of something?
winprob is more clearer to denote win probability.
Or even better is winprobpct to denote win probability in percent.

[Gian-Carlo Pascutto]

Why that configuration stuff at all? Just include the WDL with the three values for win, draw, loss in the info from engine to GUI. UCI explicitely specifies ignoring unknown things and continue parsing. If a GUI barfs on the wdl inside the info, it is not UCI-compliant anyway and should be fixed:

Yes, that's nice in theory, but in practice breaking things sucks. The other "new" UCI infos (showcurrline, showrefutations) have toggles, I assume for that reason.

[Gian-Carlo Pascutto]

Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.

There is zero chance I'd be willing to shift the decision on how to present data from the USER INTERFACE to the ENGINE.

[Gian-Carlo Pascutto]

I may add, uci protocol encourages clear key names, example,
info depth, the word depth is fully spelled out.
info score, the word score is fully spelled out too.
info multipv and others.

wdl is not clear, is it probability? is it a counter of something?
winprob is more clearer to denote win probability.
Or even better is winprobpct to denote win probability in percent.

Ok, wdl or windrawlosspct, either works and seems fine.

Splitting it in 3 and having the GUI figure out how to distribute the probability between the parts that weren't sent: not so fine.

[hgm]

I think that engine Thinking Output is by definition a table, and that there is no strong case for having a GUI display additional tables at the engine's request. I would not object to the number and titles of colums in that table to be negociable, though. Win/XBoard now displays up to 8 columns before the PV in its Engine Output window, which the user can open or close as he pleases. The columns all provide standard infos: depth, score, time, nodes, tbhits..., so that the column headers are hard-coded in the GUI. CECP Thinking Output can contain any number of numeric items without breaking backward compatibility, as the PV field is free-format text, and unexpected prefixed data will simply be displayed as part of the PV field in GUIs that do not understand the extended format. You only lose the possibility to selectively open or close the coloumns. Versions of XBoard that do understand the extended format can recognize the start of the PV, and will ignore any prefixed numeric items it does not expect. It could easily be changed to also create (hidable) columns for that, if it would only know a title for those. Unlike UCI, CECP thinking info does not contain the name of the info in any line; this is implied by the order in which they appear.

In UCI the infos are always prefixed by a keyword to specify what they are, so their order isn't (and does not have to be fixed). That would make it easy to add non-standard columns, using the info names as header. I am not sure about the parsibility, though, if not all info names can be recognized as keywords, and if the infos themselves are not always simple numbers (but can be groups of numbers, or even texts such as unit names).

[phhnguyen]

Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.

There is zero chance I'd be willing to shift the decision on how to present data from the USER INTERFACE to the ENGINE.

Those are suggestions but not decisions. When saying your data can be viewed as a table, you tell the chess GUI how to parse and update it correctly (it means you have added more information to data). However, chess GUIs and users still have the ultimate right to decide to see them or not and in which visual formats (tables, arrows, strings...).

Having more voice on display is not that bad too. Chess-GUIs and engines can cooperate in more active ways. It can solve the problem of having new data too. If an engine does any silly things and/or abuse system/chess GUIs, it will be simply rejected by the majority of users.

I see some good potentials of using JSON self-explaining data. E.g., it could help much for developing engines. A programmer can display any data he wants in good visual forms without waiting for chess GUI developers to add them. Clearly, a table is much easier to watch than looking to a lot of strings scrolling. Table's columns could be sorted too. Other visual things such as arrows, bubbles... are definitely useful. When releasing the engine, he may simply switch that data off to be more compatible with other chess GUIs.

[crem]

I agree that json is ugly, but I think using any other nicer syntax (LISP expressions?) would be worse as it's less standard.

UCI protocol is very limiting and there's no easy way to extend it.
That's why people often introduce different hacks to work around it. E.g. Lc0 interaction with Nibbler is a hack. Getting more information from stockfish also requires side-channel hacks: viewtopic.php?f=2&t=69298&p=783235#p783235.

It would be nice to have a way for a engine to output arbitrary data without risking to confuse GUIs.
It would be nice to be able to just spill out the data that is potentially useful, and then during the usage users could see which data is useful and which is not.

Even with WDL example, if we had that way to output arbitrary data, people would already use it and knew whether it's useful.

What we have instead? A hacky patch in one of forks which is not merged into master because it's non-standard, hope that it will be a "standard", and we didn't even see that feature in the wild so there's no certainty that it would be useful.


There should be the following flow of new info type:
Introduce new info type using standard method -> People trying it to find whether it's useful -> If it's useful, it's promoted to "standard" (e.g GUIs start to show it more beautifuly)
not:
Everyone hacks their own non-standard encoding of the info -> People on forum argue about format -> Engines implement it -> GUIs Implement it -> people trying it to find whether it's useful.