Stockfish, info currmove and bad UCI practice

[hgm]

I tried to run Stockfish under WinBoard + UCI2WB, to test how well the exclusion of moves during analysis works. It turned out not to work very well, and the reason seems to be that Stockfish is very tardy in sending the 'info currmove / currmovenumber' commands. Often it only starts sending these comands more than 3 sec after the search started. And to compound the disaster, it then does not send the complete set of moves, but starts somewhere in the middle (where it happened to be searching), so that you will have to wait for the next iteration (and until it finishes!) to get a complete set of moves. This means that the complete set of moves is often only available after 10-15 sec! (Note that you have to wait for a complete set from one iteration, and cannot coplete a partial set from a pprevious iteration with moves from the next, because engines often alter the move ordering from iteration to iteration.)

I noticed that other UCI engines often are also slow in sending currmove info, but those I tried were never that slow.

I know the UCI specs contain the recommendation to not spam the GUI during the first second, but 'info currmove / currmovenumber' cannot be considered spam. They send essential info (namely the set of legal moves) that the GUI has not seen before, and that it could need for issuing a UCI 'searchmoves' command. It is mindless repetition of the set could be considered spamming, but the first such set certainly isn't. So I would expect decent UCI engines to send at least one complete set of moves early on. It doesn't really have to be on the first iteration, but at least such that the GUI has received the info within the first second (and preferably within the first 200 msec).

[mar]

They send essential info (namely the set of legal moves) that the GUI has not seen before, and that it could need for issuing a UCI 'searchmoves' command.

I don't understand why the GUI should rely on currmove sent by the UCI engine? The GUI knows what position it sends to the engine and therefore it knows which moves are legal (I suppose any GUI has its own reference move generator). And it's the GUI who sends searchmoves to restrict root search so I don't see how currmove might be useful for the GUI? Perhaps I'm missing something?

[hgm]

The GUI knows what position it sends to the engine and therefore it knows which moves are legal

It doesn't if it does not know the rules of the variant that is being played. And then it also cannot have a legal move generator.

This is especially problematic in protocol adapters. They can get the instruction to exclude e2e4, but the only way to relay that to a UCI engine is to send it all other moves. While at that point the only legal move the adapter is aware of is e2e4. It might not even know the current position, because it is derived from a possibly unknown 'startpos' by a long sequence of moves with possibly unknown effects (e.g. f1c1 could have been a castling).

It seems reasonable that if an engine wants to get a list of all moves it should search, it should tell the GUI first what those moves are. Just like when it wants a position to 'go ponder' on, it does send the GUI the ponder move first.

[Uri Blass]

I believe that older versions of stockfish were more user friendly

After I mentioned in the past that stockfish does not show information about fail low the authors decided to do it less user friendly(in the past at least I could see analysis of stockfish in the first seconds and now I even cannot see it and I need to wait some seconds before I see any analysis unlike other engines and it is frustrating because I may want to decide what to do next based on looking at the analysis after one second)

Note that I do not expect the current authors to fix the problem that you mention.because it seems to me that they only care about the playing strength of stockfish.

Of course stockfish is open source so it should not be hard to make stockfish user friendly like other engines by changing the code but it seems that we need new authors who are interested to work on the code of this program to make it user friendly(I expect nothing from the current authors after they refused to fix the fact that stockfish does not show information about its fail lows).

[mar]

You're right I haven't considered variants...
I assume the problem arises when using UCI2WB. Since the adaptor knows which variant it's playing I guess the only solution (except forcing all engine authors to send info currmove often or at least full currmoves for an early iteration) is to add a move generator say for normal chess only. This would make things a lot more complicated and a bit slower but it would not rely on currmove which is considered by most as only having an informative value.

Ideally of course UCI would support exclude moves... But I guess that won't happen as (unfortunately) only very few are interested in variants.

[mar]

The question of making UCI engine analysis friendly is interesting.
I think that the engine could assume that go infinite turns analysis mode on.
In this mode the engine would ideally send all the info it can, no matter if it's searching depth 1 and it should also send output no matter if it's withing the first second of search.
That IMHO would be a good practice. In theory, it's possible that a GUI would do time management itself and use go infinite/stop to play. But I guess noone does that.
Another thing is that in analysis mode the engine should also disable contempt and use strictly symmetric eval. This would perhaps require to clear hash table when the engine switches to/from analysis mode.

[hgm]

Actually UCI seems designed on the principle that 'go infinite' does not imply analysis mode, and defines a standard option UCI_analysisMode in stead. So decisions about the things you mention (book use, eval symmetry) should really depend of the setting of that option, not on the use of 'go infinite'.

One of the things that is very annoying with UCI is that there seems to be no recognizable signal whether an engine supports 'searchmoves' or not. Apparently supporting it is mandatory, yet the actual practice is that almost no engine does. To know if an engine supports multi-PV you can at least look at if it announces the corresponding option.

[hgm]

Note that another reason for a GUI to want the currmove infos, even if it is fully rule aware, is to use them for ordering moves to present to the user for excluding or including them. So that the most relevant moves can be presented to the user at the beginning of the list. A GUI could be expected to know which moves are legal, but it could never be expected to know which moves are good or relevant.

In WinBoard, for instance, moves from the current position are presented in a header line at the top of the Engine-Output display, so that the user can include or exclude them. There only is room for a limited number of moves, though (and even if there was no such limitation, presenting all 40-50 moves would just make it difficult for the user to locate the move he wants). So you would want the relevant moves to be printed there.

Currently, there is no way WinBoard could know that. So the list starts empty, and you can only exclude the current best move there, by clicking on 'best'. This would from then on write the move that was excluded in the list, and you could go on to exclude the now best move (or other moves, you indicate by playing them on the board), which would also be added, to slowly build the list. But it might be much more convenient to start out with a non-empty list containing the leading moves in the root ordering reported by the engine. Presumably the engine will have some idea which moves are good, because they previously produced fail highs, have a high node count, or whatever, and will order those at the beginning of the list.

[mar]

Actually UCI seems designed on the principle that 'go infinite' does not imply analysis mode, and defines a standard option UCI_analysisMode in stead. So decisions about the things you mention (book use, eval symmetry) should really depend of the setting of that option, not on the use of 'go infinite'.

One of the things that is very annoying with UCI is that there seems to be no recognizable signal whether an engine supports 'searchmoves' or not. Apparently supporting it is mandatory, yet the actual practice is that almost no engine does. To know if an engine supports multi-PV you can at least look at if it announces the corresponding option.

Yes I'm aware of UCI_AnalyseMode. But I think not all GUIs support it. No big deal with go infinite i think. Should work everywhere.
As for searchmoves, you're right. There's no easy way to detect where an engine supports it.
My engine supports both searchmoves and multipv but unfortunately it doesn't report currmove when infinite search is issued. I wanted to minimize traffic that way. I was to fix that in the next version but atm i have nothing to release
Regarding standards, I saw a GUI (Arena) happily letting an engine play matches in multipv mode. Nothing wrong with that, sure. But that certainly was not intended because searchmoves and multipv are only meaningful is analysis mode.

[gladius]

Of course stockfish is open source so it should not be hard to make stockfish user friendly like other engines by changing the code but it seems that we need new authors who are interested to work on the code of this program to make it user friendly(I expect nothing from the current authors after they refused to fix the fact that stockfish does not show information about its fail lows).

Stockfish is not only open source, it is publicly hosted on Github. If you were willing to propose a patch to help analysis mode, I'm sure it would be accepted!

What is the fail-low issue that you are describing?