Winboard and arbitrary PV

[xmas79]

Hi all,
I implemented winboard interface with my engine and I have a problem showing custom PV.
From what I understood, it seems to me that WB expect at least one move in last part of the message. This is OK when you end up with exact scores or failhigh, as you can dump at least one move. But when failing low, as you don't really know what move you should do WB mixes up the messages:

This is a winboard.debug excerpt:

Code: Select all

7294 <first : 1 -110 0 453 c5d4 a5c6 b7c6 d1h5 h7g8 c3d4
7296 <first : 1 -107 0 979 c5c4 d1h5 h7g8
7296 <first : 1 -5 0 2714 h5h4 d1h5 h7g8
7297 <first : 1 0 0 3022 g7g6
7297 <first : 1 197 0 4029 d8a5 d1h5 h7g8
7298 <first : 1 197 0 4042 d8a5 d1h5 h7g8
7298 <first : 2 197 0 497 d8a5 d1h5 h7g8
7301 <first : 2 197 0 4658 d8a5 d1h5 h7g8
7309 <first : 3 197 1 11459 d8a5 d1h5 h7g8
7313 <first : 3 197 1 15781 d8a5 d1h5 h7g8
7315 <first : 4 194 2 3825 d8a5 d1h5 h7g8 h3f4 c5d4 c3d4
7320 <first : 4 194 2 10557 d8a5 d1h5 h7g8 h3f4 c5d4 c3d4
7335 <first : 5 120 4 18808 low
Impossible move d8a5, type = 0
7376 <first : 5 16 8 54590 low
Impossible move d8a5, type = 0
7770 <first : 5 -1 47 517707 g7g6 a5c6 b7c6 f2f4 d8a5 h3g5 h7h6
8234 <first : 5 -1 93 1148547 g7g6 a5c6 b7c6 f2f4 d8a5 h3g5 h7h6
9701 <first : 6 -4 240 1905887 g7g6 b2b4 c5c4 d1e2 d8h4 f2f4
10431 <first : 6 -4 313 2843546 g7g6 b2b4 c5c4 d1e2 d8h4 f2f4

and this is what winboard actually shows
7 -0.02 11.1M 0:11.36 g7g6 b2b4 c5b4 c3b4 d8e7 d1d2 f7f5 h3g5 h7h8
7 -0.02 6.5M 0:08.02 g7g6 b2b4 c5b4 c3b4 d8e7 d1d2 f7f5 h3g5 h7h8
6 -0.04 2.8M 0:03.13 g7g6 b2b4 c5c4 d1e2 d8h4 f2f4
6 -0.04 1.9M 0:02.40 g7g6 b2b4 c5c4 d1e2 d8h4 f2f4
5 +1.20 18808 0:00.04 low
5 +0.16 54590 0:00.08 low
5 -0.01 1.1M 0:00.93 g7g6 a5c6 b7c6 f2f4 d8a5 h3g5 h7h6
5 -0.01 517707 0:00.47 g7g6 a5c6 b7c6 f2f4 d8a5 h3g5 h7h6
4 +1.94 10557 0:00.02 d8a5 d1h5 h7g8 h3f4 c5d4 c3d4
4 +1.94 3825 0:00.02 d8a5 d1h5 h7g8 h3f4 c5d4 c3d4
3 +1.97 15781 0:00.01 d8a5 d1h5 h7g8
3 +1.97 11459 0:00.01 d8a5 d1h5 h7g8
2 +1.97 4658 0:00.00 d8a5 d1h5 h7g8
2 +1.97 497 0:00.00 d8a5 d1h5 h7g8
1 +1.97 4042 0:00.00 d8a5 d1h5 h7g8


The bold lines shows the problem, where you can see lines are in quite random order.

Am I missing something?


Best regards,
Natale.

[hgm]

I don't think you need at least one move. The PV in WB protocol is free-format text, and an empty string should be acceptable.

Note that some of the newer WinBoard functions try to interpret the PV you send, however. (E.g. if you want to play out the PV on the board, or advance the game along it in analyze mode.) It uses the PGN parser for this, so if you want to display non-moves, it wold be best to use a format that is valid PGN. (Like putting {braces} around any remarks.)

The order is not random, but sorted by score. This is usually the way you would want them in multi-PV mode. Normally (i.e. in single-PV mode) PVs arrive in order of increasing score. It seems that this backfires when you want to report fail lows, where the initial score is a high upper bound, and a lower exact score will follow when it is resolved. For a fail high there would be no problem, as the resolved scores would be higher than the lower bound fom the failed search.

I am afraid there is no solution for this without some new protocol to indicate that the initial 1.20 is not a score but a fail-low upper bound, and that scores received later should never be sorted below it, even if they are lower. I guess a good way to do this would be to use a question mark as PV in case of a fail low. A fail high cold be indicated by an exclamation point behind the PV (which normally would be just a single move). This could be useful in case search instability would cause a resolved score below the initial lower bound.

[xmas79]

Ahh yes, I see... I now noticed it happens also when engine fails high and then gets back a score lower than the fail high.

A simple extension to the existing protocol in the next Xboard/Winboard release would do the job: an engine could send in the "feature" string a new keyword "multipv=0" to disable this kind of sorting, while xboard/winboard assumes multipv=1 by default. I guess it would be very simple patch.

Thanks,
Natale.

[hgm]

I would think that would cause a problem for engines that both report fail lows and support multi-PV...

I think it would be much better if the engines simply identified fail lows by a question mark. That would not break their multi-PV operation.

[xmas79]

I would think that would cause a problem for engines that both report fail lows and support multi-PV...

I think it would be much better if the engines simply identified fail lows by a question mark. That would not break their multi-PV operation.

Are they working well now?

[hgm]

That depends on what you mean by 'they'. Engines that do multi-PV and do not report fail lows do work well.

No existing engine would work well if working well required feature multipv=0, as no such engine sends that, and WinBoard does not implement it yet. To make reporting of fail lows 'work well' would always require a modification in both the engine and WinBoard.

Better to make it a modification that would not cause problems in other areas...

[xmas79]

What I propose is then to take care of that and make it work for every kind of engine.

I would expect the lines organized as follow:

single-pv engines: sorted by depth, then sorted by time. This is the only way to see what actually the engine is thinking for these kind of engines.

multiPV: the "sort by score" way seems to me to be the best. I don't do multiPV, so I really don't know if there's a better way. A would actually think that a multiPV engine will send multiPV lines in random order, so no assumptions can be done about FH/FL, exact scores, right? I'm unware on how many engines report a new PV as soon they find a new PV, or wait the iteration to complete and then print all PVs togheter. Is there a standard for this?


As I don't believe we have a standard here, I think that as you already said an engine should report if the PV that has been reported is an exact score or a bound. What you propose seems very logical: the first char of the PV can be "!" for a FH, "?" for a FL, and I add an "=" for an exact score. Winboard could look at this char and then sort by "group" of that char, and then inside that group by score. The problem is that when you have a single PV engine that reports multiple FL, all these lines have the same "?" in front, so sorting by scores won't work. MultiPV engine will have the same problem. So I would say that winboard should really know if the engine is a singlePV engine or a multiPV engine... Right?

Just my 2 cents.
Natale.

[hgm]

There is unfortunately no standard at all for how engines implement multi-PV. 'Fail low' also bcomes a sort of fuzzy concept with multi-PV, because what is a fail low for one of the moves, could very well be a fail high for others.

I did not want to consider fail-low lines as a group to be sorted by score, for exactly the reason you give. Engines that open the window in steps might get a number of fail lows of successively lower score.

I guess a fail low or fail high would always be followed by a re-search to resolve it (perhaps after several more fails). So basically the score of any failed search should, for the purpose of sorting, be set to the value of the first non-failing search that follows it. So that temporally adjacent groups (fail, fail, exact) would always be kept as a single unit.

Btw, it seems more logical to have the ! on a fail high at the end than at the beginning. The PV of a fail high should contain exactly a single move, and the normal PGN meaning of the ! annotation applies quite naturally to it.