UCI Win/Draw/Loss reporting

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Ras
Posts: 1161
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: UCI Win/Draw/Loss reporting

Post by Ras » Wed Oct 23, 2019 8:00 pm

hgm wrote:
Wed Oct 23, 2019 6:52 pm
if we would get dozens of new infos
That YAGNI will hardly make it across many engines or GUIs anyway. WDL is different because NN engines like LC0 have a different way of expressing the evaluation - not in centipawns, but in terms of probability. That's a fundamentally different kind of strong chess engine that does need some amendments.

Actually, the cleanest way would be to enhance the "score" tag to allow "score wdl X Y Z" with X Y Z in permill. The sum of all must be 1000. That's easy to implement and robust to parse. If that is triggered by a UCI_WdlScore boolean (default: false), that allows for a smooth and simple integration.

Drowning the protocol in enhancements doesn't pan out that well because that ends up with one or two engines and one GUI that offer full support.
Rasmus Althoff
https://www.ct800.net

User avatar
phhnguyen
Posts: 397
Joined: Wed Apr 21, 2010 2:58 am
Location: Australia
Full name: Nguyen Hong Pham
Contact:

Re: UCI Win/Draw/Loss reporting

Post by phhnguyen » Wed Oct 23, 2019 11:55 pm

Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.

Table
1) Data (partly command)
- A row of a table requires data as an array. Each item has a column name and value. Value could be an integer, a real number or a string.

Code: Select all

["W": 60, "D": 20, "L": 20]
- A table may have few rows - an array of arrays:

Code: Select all

[["Ord": 5, "W": 60, "D": 20, "L": 20], ["Ord": 2, "W": 80, "D": 5, "15": 20]]
2) Table of 1 row (full command)

Code: Select all

uci json {"table": ["W": 60, "D": 20, "L": 20]}
If chess GUIs receive new data of the table, they will replace old table by new one.


3) Table of multi-rows

Code: Select all

uci json {"table": [["Ord": 1, "W": 60, "D": 20, "L": 20], ["Ord": 2, "W": 40, "D": 60, "L": 30]]}
If chess GUIs receive new data, they will replace old table by new one.

4) Table of multi-rows, update on-fly

We need a special column, named "depth", "row" or "move".

Code: Select all

uci json {"table": ["row": 1, "W": 60, "D": 20, "L": 20]}
Example data for Lc0:

Code: Select all

uci json {"table": ["move": "f2f4", "self depth": 351, "N": "33 (+ 3)", "P%": 2.98, "Q": -0.07531, "D": 0.209, "U": 0.18615, "Q+U": 0.11084, "V": -0.0719] }
If chess GUIs receive new data, they will replace the corresponding row (based on values of column "row", "depth", "move") in the table by new row. Column "depth" may be sorted descently when "row" ascently.

5) Multi tables

Tables may be distinguished by their names, the data format should be a map. Each time the engine should send data of one table only (not all multi-table data at once).

Code: Select all

uci json {"table": {"name" : "WDL stats", "data": ["row": 1, "W": 60, "D": 20, "L": 20]}}

Notes:
- The data structures to display tables are quite flexible, depend on needs.
- Reserved/key words: "table", "row", "depth", "move"
- All tables will be reset when side-to-move of the board changed

Chess GUIs may display those tables separately with other info and let users decide to view or hide them (e.g., via buttons)

Image
Last edited by phhnguyen on Thu Oct 24, 2019 12:05 am, edited 2 times in total.
https://banksiagui.com
A freeware chess GUI, based on opensource Banksia - the chess tournament manager

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: UCI Win/Draw/Loss reporting

Post by Ferdy » Thu Oct 24, 2019 12:00 am

Gian-Carlo Pascutto wrote:
Wed Oct 23, 2019 2:47 pm
Ferdy wrote:
Wed Oct 23, 2019 2:09 pm
Perhaps simpler and more uci way of communicating is by key value pair. Example,

Code: Select all

winprob 20 drawprob 60 lossprob 20
values like 20, 60 and 20 are in percent.
I don't see the benefit of having extra tags. We don't send a pv either with pv e4e4 pv e7e5 pv g1f3. I also don't see how it is simpler, looks the opposite to me.
In this particular case extra tag is appropriate since winprob can stand on its own. If position is already winning sending winprob 99 is simpler. There is no need to send the drawprob and lossprob. Most games are drawish, sending drawprob 99 is also simpler, most users are no longer concern of winprob and lossprob.

Regarding pv it is a different case, pv line is better presented as a sequence of moves.

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: UCI Win/Draw/Loss reporting

Post by Ferdy » Thu Oct 24, 2019 3:36 am

Ferdy wrote:
Thu Oct 24, 2019 12:00 am
Gian-Carlo Pascutto wrote:
Wed Oct 23, 2019 2:47 pm
Ferdy wrote:
Wed Oct 23, 2019 2:09 pm
Perhaps simpler and more uci way of communicating is by key value pair. Example,

Code: Select all

winprob 20 drawprob 60 lossprob 20
values like 20, 60 and 20 are in percent.
I don't see the benefit of having extra tags. We don't send a pv either with pv e4e4 pv e7e5 pv g1f3. I also don't see how it is simpler, looks the opposite to me.
In this particular case extra tag is appropriate since winprob can stand on its own. If position is already winning sending winprob 99 is simpler. There is no need to send the drawprob and lossprob. Most games are drawish, sending drawprob 99 is also simpler, most users are no longer concern of winprob and lossprob.

Regarding pv it is a different case, pv line is better presented as a sequence of moves.
I may add, uci protocol encourages clear key names, example,
info depth, the word depth is fully spelled out.
info score, the word score is fully spelled out too.
info multipv and others.

wdl is not clear, is it probability? is it a counter of something?
winprob is more clearer to denote win probability.
Or even better is winprobpct to denote win probability in percent.

Gian-Carlo Pascutto
Posts: 1185
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: UCI Win/Draw/Loss reporting

Post by Gian-Carlo Pascutto » Thu Oct 24, 2019 7:42 am

Ras wrote:
Wed Oct 23, 2019 6:04 pm
Why that configuration stuff at all? Just include the WDL with the three values for win, draw, loss in the info from engine to GUI. UCI explicitely specifies ignoring unknown things and continue parsing. If a GUI barfs on the wdl inside the info, it is not UCI-compliant anyway and should be fixed:
Yes, that's nice in theory, but in practice breaking things sucks. The other "new" UCI infos (showcurrline, showrefutations) have toggles, I assume for that reason.

Gian-Carlo Pascutto
Posts: 1185
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: UCI Win/Draw/Loss reporting

Post by Gian-Carlo Pascutto » Thu Oct 24, 2019 7:46 am

phhnguyen wrote:
Wed Oct 23, 2019 11:55 pm
Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.
There is zero chance I'd be willing to shift the decision on how to present data from the USER INTERFACE to the ENGINE.

Gian-Carlo Pascutto
Posts: 1185
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: UCI Win/Draw/Loss reporting

Post by Gian-Carlo Pascutto » Thu Oct 24, 2019 7:50 am

Ferdy wrote:
Thu Oct 24, 2019 3:36 am
I may add, uci protocol encourages clear key names, example,
info depth, the word depth is fully spelled out.
info score, the word score is fully spelled out too.
info multipv and others.

wdl is not clear, is it probability? is it a counter of something?
winprob is more clearer to denote win probability.
Or even better is winprobpct to denote win probability in percent.
Ok, wdl or windrawlosspct, either works and seems fine.

Splitting it in 3 and having the GUI figure out how to distribute the probability between the parts that weren't sent: not so fine.

User avatar
hgm
Posts: 23725
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: UCI Win/Draw/Loss reporting

Post by hgm » Thu Oct 24, 2019 8:51 am

I think that engine Thinking Output is by definition a table, and that there is no strong case for having a GUI display additional tables at the engine's request. I would not object to the number and titles of colums in that table to be negociable, though. Win/XBoard now displays up to 8 columns before the PV in its Engine Output window, which the user can open or close as he pleases. The columns all provide standard infos: depth, score, time, nodes, tbhits..., so that the column headers are hard-coded in the GUI. CECP Thinking Output can contain any number of numeric items without breaking backward compatibility, as the PV field is free-format text, and unexpected prefixed data will simply be displayed as part of the PV field in GUIs that do not understand the extended format. You only lose the possibility to selectively open or close the coloumns. Versions of XBoard that do understand the extended format can recognize the start of the PV, and will ignore any prefixed numeric items it does not expect. It could easily be changed to also create (hidable) columns for that, if it would only know a title for those. Unlike UCI, CECP thinking info does not contain the name of the info in any line; this is implied by the order in which they appear.

In UCI the infos are always prefixed by a keyword to specify what they are, so their order isn't (and does not have to be fixed). That would make it easy to add non-standard columns, using the info names as header. I am not sure about the parsibility, though, if not all info names can be recognized as keywords, and if the infos themselves are not always simple numbers (but can be groups of numbers, or even texts such as unit names).

User avatar
phhnguyen
Posts: 397
Joined: Wed Apr 21, 2010 2:58 am
Location: Australia
Full name: Nguyen Hong Pham
Contact:

Re: UCI Win/Draw/Loss reporting

Post by phhnguyen » Thu Oct 24, 2019 9:55 am

Gian-Carlo Pascutto wrote:
Thu Oct 24, 2019 7:46 am
phhnguyen wrote:
Wed Oct 23, 2019 11:55 pm
Let me suggest a description about self-explaining data in which engines can tell chess GUIs how to display their data in the form of tables.
There is zero chance I'd be willing to shift the decision on how to present data from the USER INTERFACE to the ENGINE.
Those are suggestions but not decisions. When saying your data can be viewed as a table, you tell the chess GUI how to parse and update it correctly (it means you have added more information to data). However, chess GUIs and users still have the ultimate right to decide to see them or not and in which visual formats (tables, arrows, strings...).

Having more voice on display is not that bad too. Chess-GUIs and engines can cooperate in more active ways. It can solve the problem of having new data too. If an engine does any silly things and/or abuse system/chess GUIs, it will be simply rejected by the majority of users.

I see some good potentials of using JSON self-explaining data. E.g., it could help much for developing engines. A programmer can display any data he wants in good visual forms without waiting for chess GUI developers to add them. Clearly, a table is much easier to watch than looking to a lot of strings scrolling. Table's columns could be sorted too. Other visual things such as arrows, bubbles... are definitely useful. When releasing the engine, he may simply switch that data off to be more compatible with other chess GUIs.
https://banksiagui.com
A freeware chess GUI, based on opensource Banksia - the chess tournament manager

crem
Posts: 125
Joined: Wed May 23, 2018 7:29 pm

Re: UCI Win/Draw/Loss reporting

Post by crem » Thu Oct 24, 2019 1:31 pm

I agree that json is ugly, but I think using any other nicer syntax (LISP expressions?) would be worse as it's less standard.

UCI protocol is very limiting and there's no easy way to extend it.
That's why people often introduce different hacks to work around it. E.g. Lc0 interaction with Nibbler is a hack. Getting more information from stockfish also requires side-channel hacks: viewtopic.php?f=2&t=69298&p=783235#p783235.

It would be nice to have a way for a engine to output arbitrary data without risking to confuse GUIs.
It would be nice to be able to just spill out the data that is potentially useful, and then during the usage users could see which data is useful and which is not.

Even with WDL example, if we had that way to output arbitrary data, people would already use it and knew whether it's useful.

What we have instead? A hacky patch in one of forks which is not merged into master because it's non-standard, hope that it will be a "standard", and we didn't even see that feature in the wild so there's no certainty that it would be useful.


There should be the following flow of new info type:
Introduce new info type using standard method -> People trying it to find whether it's useful -> If it's useful, it's promoted to "standard" (e.g GUIs start to show it more beautifuly)
not:
Everyone hacks their own non-standard encoding of the info -> People on forum argue about format -> Engines implement it -> GUIs Implement it -> people trying it to find whether it's useful.

Post Reply