Appeal to authors: Standardize kibitz format

[hgm]

The next WinBoard, when running in zippy mode, as a part of the -autoKibitz feature, will try to parse engine kibitz lines from the opponent. This to extract depth and score, so that they can appear in the PGN file (and be used for the evaluation graph) just like the depth and score of your own, local engine.

Now for this to work, WinBoard must be able to peel the depth and score out of the kibitz line. Of course it will recognize the format in which it autoKibitzes the thinking output itself, which starts with depth/score, e.g. 12/+1.27 , followed by other stuff (which will not appear in the PGN anyway).

Now not everyone playing on line uses the WB -autoKibitz. Some like their own engines to kibitz, so they can kibitz other info than WB would, and some do not even use WB. Now for the -autoKibitz option to be most useful for those who do use it, it would be very important that WB can recognize the depth and score info from your engine.

So I want to make this appeal to all authors:

Please kibitz in a format that starts with depth/score (in %d/%+.2f format, for C programmers). And do this only once for each move, just after you sent the move to which that info applies.

If you want to give multiple kibitzes per move, or kibitz during ponder, please use another format, so that it cannot confuse WB. E.g., if you like the depth/score format, preceed it with another (non-blank) character on all thinking info that did not apply to the move that was just played.

Your opponent will be so grateful for this. (And might even be willing to mail you a PGN annotated with scores of both engines afterwards!)

I picked the format depth/score for several reasons:
1) it is how it appears in th PGN comments
2) it is very compact, and does not waste space on the line to keywords like 'score=' and 'depth=', so more space is left for other info (like long PVs).

Currently, WB does not store PVs in the PGN. Just in case we would ever want that, I recommend putting the PV at the end of the kibitz line, preceeded by the string "PV=", so the PV can remain free format, and WB can simply capture everything between the PV= and the new-line.

[sje]

Symbolic's ICS kibitz format (samples)

[ <engine POV score (pawns)> / <depth> / <node count> ] <PV>

Code: Select all

[+0.672/1/8] 15 Nc3
[+0.872/2/271] 15 Nc3 dxc5 16 Qxc5
[+0.872/3/816] 15 Nc3 dxc5 16 Qxc5
[+0.824/4/7,393] 15 Nc3 dxc5 16 Qxc5 Be6
[+0.726/5/28,587] 15 Nc3 dxc5 16 Qxc5 Bg4 17 Qd6 Bxf3 18 gxf3
[+0.706/6/250,377] 15 Nc3 dxc5 16 Qxc5 Bd7 17 Qd6 Bb6
[+0.732/6/276,868] 15 Nb4 dxc5 16 Qxc5 Qxe4 17 Rfe1 Qf4 18 Nxc6
[+0.669/7/936,664] 15 Nb4 dxc5 16 Qxc5 Bb7 17 e5 Bb6 18 Qc4

Code: Select all

[-0.641/1/22] 15... dxc5 16 Qxc5 Qxe4 17 Qxc6
[-0.641/2/168] 15... dxc5 16 Qxc5 Qxe4 17 Qxc6
[-0.641/3/534] 15... dxc5 16 Qxc5 Qxe4 17 Qxc6
[-0.641/4/4,308] 15... dxc5 16 Qxc5 Qxe4 17 Qxc6
[-0.732/5/13,235] 15... dxc5 16 Qxc5 Qxe4 17 Rfe1 Qf4 18 Nxc6
[-0.669/6/118,430] 15... dxc5 16 Qxc5 Bb7 17 e5 Bb6 18 Qc4
[-0.707/7/367,310] 15... dxc5 16 Qxc5 Bb7 17 e5 Bc7 18 Rad1 Qe6
[-0.649/8/3,004,124] 15... dxc5 16 Qxc5 Bb7 17 e5 Bc7 18 Nd5 cxd5 19 Qxc7

[hgm]

Indeed, this is a good example of what I mean. This would not be recognized by WinBoard because of the preceeding '['. Which is a good thing, because it is also kibitzed before the move, during thinking, and thus does not apply to the search that produced the move that was eventually played.

So what I ask is to printthe output for the move that you finally play without the'preceeding '[', after you played the move, so that it can go into your opponent's PGN.

Note that in my first posting I got it wrong.

Standard in the PGN is to have the score before the slash, and the depth after it, like Symbolic does. Currently WB autoKibitzes this the wrong way around. I will fix this in the next release.

So the recommended format is score/depth ("%+.2f/%d ...").

[bob]

The next WinBoard, when running in zippy mode, as a part of the -autoKibitz feature, will try to parse engine kibitz lines from the opponent. This to extract depth and score, so that they can appear in the PGN file (and be used for the evaluation graph) just like the depth and score of your own, local engine.

Now for this to work, WinBoard must be able to peel the depth and score out of the kibitz line. Of course it will recognize the format in which it autoKibitzes the thinking output itself, which starts with depth/score, e.g. 12/+1.27 , followed by other stuff (which will not appear in the PGN anyway).

Now not everyone playing on line uses the WB -autoKibitz. Some like their own engines to kibitz, so they can kibitz other info than WB would, and some do not even use WB. Now for the -autoKibitz option to be most useful for those who do use it, it would be very important that WB can recognize the depth and score info from your engine.

So I want to make this appeal to all authors:

Please kibitz in a format that starts with depth/score (in %d/%+.2f format, for C programmers). And do this only once for each move, just after you sent the move to which that info applies.

If you want to give multiple kibitzes per move, or kibitz during ponder, please use another format, so that it cannot confuse WB. E.g., if you like the depth/score format, preceed it with another (non-blank) character on all thinking info that did not apply to the move that was just played.

Your opponent will be so grateful for this. (And might even be willing to mail you a PGN annotated with scores of both engines afterwards!)

I picked the format depth/score for several reasons:
1) it is how it appears in th PGN comments
2) it is very compact, and does not waste space on the line to keywords like 'score=' and 'depth=', so more space is left for other info (like long PVs).

Currently, WB does not store PVs in the PGN. Just in case we would ever want that, I recommend putting the PV at the end of the kibitz line, preceeded by the string "PV=", so the PV can remain free format, and WB can simply capture everything between the PV= and the new-line.

The idea is good, but the implementation is bad. Why not choose a more readable kibitz format, and then extract what you want? something like

14/3.2 (or even worse when programs show the a/b form of depth)

is not readable yet that is exactly what kibitzing is supposed to be used for. This will only invite lots of questions everytime someone observes a game where a computer kibitzes in that form.

Why not use something that won't generate additional questions. I agree that a standard kibitz would be nice so that all programs follow the same standard. However, how are you going to tell all programs to kibitz, and what to kibitz? Even that is non-standard at the moment... Crafty can be told to display one kibitz per move (kibitz=4), one kibitz at the end of each search iteration (kibitz=5) or one kibitz any time the PV changes, including fail highs (kibitz=6).

[brianr]

How about a sort of EPD/FEN for kib info with a "standard" first part with the minimal "essential" information like score, depth (basic "full-ply search"), pv, and then an optional part with some identification tokens, like egtb probes/hits, extended depth, or whatever? The optional parts could be somewhat standardized, like "bm", "am", "id", etc.

Of course, the score should be from the engine's POV, not white's

[hgm]

The idea is good, but the implementation is bad. Why not choose a more readable kibitz format, and then extract what you want? something like

14/3.2 (or even worse when programs show the a/b form of depth)

is not readable yet that is exactly what kibitzing is supposed to be used for. This will only invite lots of questions everytime someone observes a game where a computer kibitzes in that form.

...

Isn't this a bit of an exaggeration? For one, it is the same notation as used in the PGN. So people that are used to reading PGN are very familiar with it.

In the second place, ICS communication, although readable text, should be considered as low-leve protocol for client-server communication. As such it is not really meant for reading by Humas, especialy not uninitated Humans. The engine kibitzes are not really an excepton to this. The client (i.e. WinBoard / XBoard) should grab them, digest them, and present the info in a more palatable form to the user. For instance by displaying the scores in the eval graph, or show the depth/score in the game history window as part of the PGN comments.

[bob]

How about a sort of EPD/FEN for kib info with a "standard" first part with the minimal "essential" information like score, depth (basic "full-ply search"), pv, and then an optional part with some identification tokens, like egtb probes/hits, extended depth, or whatever? The optional parts could be somewhat standardized, like "bm", "am", "id", etc.

Of course, the score should be from the engine's POV, not white's

Fall thru your butt and hang yourself.



You can keep wishing for that engine POV from crafty, but to quote Burgess Merideth in Grumpy Old Men, "You can wish in one hand, crap in the other, and see which one fills up first."

[bob]

The idea is good, but the implementation is bad. Why not choose a more readable kibitz format, and then extract what you want? something like

14/3.2 (or even worse when programs show the a/b form of depth)

is not readable yet that is exactly what kibitzing is supposed to be used for. This will only invite lots of questions everytime someone observes a game where a computer kibitzes in that form.

...

Isn't this a bit of an exaggeration? For one, it is the same notation as used in the PGN. So people that are used to reading PGN are very familiar with it.

In the second place, ICS communication, although readable text, should be considered as low-leve protocol for client-server communication. As such it is not really meant for reading by Humas, especialy not uninitated Humans. The engine kibitzes are not really an excepton to this. The client (i.e. WinBoard / XBoard) should grab them, digest them, and present the info in a more palatable form to the user. For instance by displaying the scores in the eval graph, or show the depth/score in the game history window as part of the PGN comments.

No it isn't the same. Chessbase has done more than one version of this, creating _new_ PGN tags for eval, etc. Which makes a PGN very difficult to read. But my comment came from ICC where I used to not label the values in kibitzes and got continual questions.

But there is no standard for PGN that I am aware of, with respect to how program information is encapsulated, other than it has to be done inside comments.

It sounds like you are trying to add a new engine-to-GUI command, which would be fine by me. Just tell me how to format it and I'll add that quickly. I don't like the idea of your trying to parse the "tellics kibitz <etc>" string stuff since I have changed mine more than once as I add things that provide new information that observers might find interesting, such as book moves, and such.

[sje]

Tangent: kibitz fault with xboard 4.3.15

In a recent ICC game, Symbolic failed to emit kibitz data to the ICS. I hadn't seen this before and hadn't modified the kibitz trigger code.

Usually, xboard sends "random", "ics <ICS name>", and "hard" in quick succession. The ICS name is "-" when there's no connection, and Symbolic will only emit ICS kibitz data if there is a connection.

Well, for unknown reason, xboard did not send an "ics" to Symbolic in a recent ICS game, so Symbolic did not kibitz. The other commands in the new game prep sequence all came within a millisecond or so of each other as is the usual case. But somehow, there was no "ics".

I have not been able to reproduce this bug.

[adams161]

with regard to the kibitzes being readable, the motivation is to put evals in pgn? can thinking output be used? as to regard to icc, as robert hyatt mentions we want info that we want displayed to others or us watching the bot. there is an ics argument to send to winboard. winboard will tell you you are on an ics. does it make sense to kibtiz on an ics in the way you want others to see and when not on an ics, if you still even kibitz, in a way you want winboard to read? help me undertand this.