Stockfish, info currmove and bad UCI practice

[mcostalba]

Of course stockfish is open source so it should not be hard to make stockfish user friendly like other engines by changing the code but it seems that we need new authors who are interested to work on the code of this program to make it user friendly(I expect nothing from the current authors after they refused to fix the fact that stockfish does not show information about its fail lows).

Stockfish is not only open source, it is publicly hosted on Github. If you were willing to propose a patch to help analysis mode, I'm sure it would be accepted!

What is the fail-low issue that you are describing?

It is something that has been fixed months ago and is fixed already in 2.2.2, but please don't remove from people the subtle pleasure to say cheap bul...ts

[Don]

I tried to run Stockfish under WinBoard + UCI2WB, to test how well the exclusion of moves during analysis works. It turned out not to work very well, and the reason seems to be that Stockfish is very tardy in sending the 'info currmove / currmovenumber' commands. Often it only starts sending these comands more than 3 sec after the search started. And to compound the disaster, it then does not send the complete set of moves, but starts somewhere in the middle (where it happened to be searching), so that you will have to wait for the next iteration (and until it finishes!) to get a complete set of moves. This means that the complete set of moves is often only available after 10-15 sec! (Note that you have to wait for a complete set from one iteration, and cannot coplete a partial set from a pprevious iteration with moves from the next, because engines often alter the move ordering from iteration to iteration.)

I noticed that other UCI engines often are also slow in sending currmove info, but those I tried were never that slow.

I know the UCI specs contain the recommendation to not spam the GUI during the first second, but 'info currmove / currmovenumber' cannot be considered spam. They send essential info (namely the set of legal moves) that the GUI has not seen before, and that it could need for issuing a UCI 'searchmoves' command. It is mindless repetition of the set could be considered spamming, but the first such set certainly isn't. So I would expect decent UCI engines to send at least one complete set of moves early on. It doesn't really have to be on the first iteration, but at least such that the GUI has received the info within the first second (and preferably within the first 200 msec).

I believe a GUI should have it's own move generator and not assume anything. If an engine plays an illegal move, how do you adjudicate the win? If any engine says it has won when it hasn't, what can you do?

I can understand however that for a GUI that must easily be able to support a large variety of games, it might be more feasible to rely on some help from the engines.

[zamar]

I think that currently the only purpose of currmovenumber is to give user an indication of the progress.

However this idea is broken by design, because in multithread mode SF (and maybe also other engines?) search several moves simultaneously.

[hgm]

I believe a GUI should have it's own move generator and not assume anything. If an engine plays an illegal move, how do you adjudicate the win? If any engine says it has won when it hasn't, what can you do?

Usually the opponent engine would refuse the move, if it is illegal. So even in engine-engine matches a GUI would be able to detect one of the engines must be malfunctioning. It just doesn't know which one. But it could terminate the game, and save it with 'unfinished' result, and a result comment like "{Aborted due to engine disagreement}". The user could later filter out games that ended this way, and decide who was in the wrong. At some point he would have to look at it anyway, to fix the engine, so this is really not much of a problem.

I can understand however that for a GUI that must easily be able to support a large variety of games, it might be more feasible to rely on some help from the engines.

WinBoard protocol evolved from the GNU Chess text interface, which was designed for GUI-less human-engine games, where the engine carries all knowledge, and has to discipline the human opponent. Because of this it has many features to let the engine take control: the engine can decide if the moves are legal, and gives to-the-point error messages when they are not, the engine can decide the game has ended, and how... For engine-engine matches with buggy engines that might not be sufficient, but the entire world does not revolve around engine-engine games, and even there in most cases the engine's judgement about move legality can be trusted. Human-engine and analysis have large market shares, and a design where the engine runs the show completely is entirely satisfactory there.

For flexibility it has a huge advantage to leave the engine in control of the rules. Basically it means that you can play arbitrary variants without any change in the GUI, just by switching engine. The WinBoard Alien Edition is the biggest step in the direction of a completely knowledgeless GUI so far, and can be used to play Checkers as easily as Chess, Go, Reversi or Ultima. Without losing the ability to have the possible to-squares for a piece that you pick up marked by yellow and red dots, or the pieces that would disappear on capture by green dots, have one-click moving for pieces with only a single legal move, etc.

[Don]

I think that currently the only purpose of currmovenumber is to give user an indication of the progress.

However this idea is broken by design, because in multithread mode SF (and maybe also other engines?) search several moves simultaneously.

Good point - I agree.

In fact, both protocols are broken with assumptions about computer chess program that do not necessarily have to be true. For example a program "must" iterate in order to produce a move. If someone decides to build a program based on the new MCTS methods used in GO, it would not translate naturally to a chess GUI.

By the same token, a lot of things are not included in GUI's that humans usually have access to, such as who their opponent is and what his rating is.

[hgm]

By the same token, a lot of things are not included in GUI's that humans usually have access to, such as who their opponent is and what his rating is.

WinBoard tells the engine who its opponent is, and which rating he has...

[Don]

I believe a GUI should have it's own move generator and not assume anything. If an engine plays an illegal move, how do you adjudicate the win? If any engine says it has won when it hasn't, what can you do?

Usually the opponent engine would refuse the move, if it is illegal. So even in engine-engine matches a GUI would be able to detect one of the engines must be malfunctioning. It just doesn't know which one. But it could terminate the game, and save it with 'unfinished' result, and a result comment like "{Aborted due to engine disagreement}". The user could later filter out games that ended this way, and decide who was in the wrong. At some point he would have to look at it anyway, to fix the engine, so this is really not much of a problem.

Of course there are work-arounds, but not ones that are very elegant. I want my GUI to tell me that program X lost the game due to trying to execute an illegal move.

You would never do a similar thing with the clock, saying that program X thinks program Y lost due to time forfeit even though program Y does not agree. The point is that the GUI should not be passing off adjudication chores it can do for itself.

I can understand however that for a GUI that must easily be able to support a large variety of games, it might be more feasible to rely on some help from the engines.

WinBoard protocol evolved from the GNU Chess text interface, which was designed for GUI-less human-engine games, where the engine carries all knowledge, and has to discipline the human opponent. Because of this it has many features to let the engine take control: the engine can decide if the moves are legal, and gives to-the-point error messages when they are not, the engine can decide the game has ended, and how... For engine-engine matches with buggy engines that might not be sufficient, but the entire world does not revolve around engine-engine games, and even there in most cases the engine's judgement about move legality can be trusted. Human-engine and analysis have large market shares, and a design where the engine runs the show completely is entirely satisfactory there.

For flexibility it has a huge advantage to leave the engine in control of the rules. Basically it means that you can play arbitrary variants without any change in the GUI, just by switching engine. The WinBoard Alien Edition is the biggest step in the direction of a completely knowledgeless GUI so far, and can be used to play Checkers as easily as Chess, Go, Reversi or Ultima. Without losing the ability to have the possible to-squares for a piece that you pick up marked by yellow and red dots, or the pieces that would disappear on capture by green dots, have one-click moving for pieces with only a single legal move, etc.

I have long considered creating a more rational and logical protocol for Komodo and just having a translation library for UCI and WINBOARD. It's a lot of work however and I'm not motivated enough to go to this trouble when either protocol is "good enough." If I ever create my own GUI however, I will create a new protocol to go with it that follows my own philosophy and of course it would have UCI and WINBOARD built in as deprecated protocols.

[BubbaTough]

By the same token, a lot of things are not included in GUI's that humans usually have access to, such as who their opponent is and what his rating is.

I hate this. Even when a protocol supports this, most people don't utilize it (home users, testers, etc.). There is a lot of interesting and fun work to be done in the area of molding your moves / draw offers to your opponent, that is subtly discouraged by the community. Only a few online addicts (like Tinker that plays on ICC a lot) seem to do much here, because on ICC you can ensure this information gets to your engine.

-Sam

[Don]

By the same token, a lot of things are not included in GUI's that humans usually have access to, such as who their opponent is and what his rating is.

WinBoard tells the engine who its opponent is, and which rating he has...

That is a good thing - I really wish UCI had that.

[Don]

By the same token, a lot of things are not included in GUI's that humans usually have access to, such as who their opponent is and what his rating is.

I hate this. Even when a protocol supports this, most people don't utilize it (home users, testers, etc.). There is a lot of interesting and fun work to be done in the area of molding your moves / draw offers to your opponent, that is subtly discouraged by the community. Only a few online addicts (like Tinker that plays on ICC a lot) seem to do much here, because on ICC you can ensure this information gets to your engine.

-Sam

Yes, and it turns out that it's very important to set your contempt factor pretty high when playing an opponent much weaker than yourself.

Can you imagine Carlsen accepting a draw from you or I because we managed to get out of book with a slight advantage (because we got the white pieces?) Programs typically have contempt factors close to zero and yet in the same scenario would happily take a draw playing hundreds of ELO down.