some UCI protocol issues/questions

[CThinker]

I often will take a list of 150 positions, and let a very strong engine analyze at (for instance) 1/2 hour per position over the weekend. It will often be the case that ten hours or so is completely wasted because of sitting there doing nothing for 20 of the positions.

I don't get it. Why not use a timecontrol? If you're using infinite analysis, you get exactly what you ask for.

He is using time control - 30 minutes per position, for 150 positions. Not infinite analysis.

The problem is that, with UCI, the engine will use of all of the time control even if the solution is reached in less time (like in a mate).

So, let me simplify Dan's point with an example...

Let us say I want to automate the anlysis of 3 positions, at 1 hour per position. Let us also say that one of the positions is a mate in 10 moves, and can be solved in 1 minute.

With UCI, the entire process will take 3 hours. With Winboard, the entire process will take 2 hours and 1 minute.

[CThinker]

Like many other things, it is a big deal for me and for almost nobody else on the planet.

I often will take a list of 150 positions, and let a very strong engine analyze at (for instance) 1/2 hour per position over the weekend. It will often be the case that ten hours or so is completely wasted because of sitting there doing nothing for 20 of the positions.

It is especially true when analyzing a large segment of endgame positions.

It probably bothers nobody else at all, but it bugs me to no end.

It seems to me that you bring this upon yourself. If you want to do not exactly 1/2 hour per position, but 1/2 hour maximum, you should use the engine not in analysis, but in fixed-max-time-per-move mode (st in WB, movetime in UCI). Then the engine will terminate the search if there is nothing left to search.

I think Dann is being misunderstood here.

Here is an example of a Winboard sequence:
setboard POSITION1
tc 6000 // 1 minute
go
setboard POSITION2
tc 6000 // 1 minute
go
setboard POSITION3
tc 6000 // 1 minute
go

That entire process will take at most 3 minutes.

However, if POSITION1 and POSITON2 happens to be mates and can be solved in 3 seconds each, then the entire process will just take 1 minute and 6 seconds.

But with UCI, the entire process will always take 3 minutes, no matter what the positions are.

[CThinker]

I have never seen this issue with Winboard engines.

Here is an example using Crafty. Notice how there is search progress when analyzing from the initial chess position. But with mate positions, Crafty simply stops searching (no CPU usage).

Not all engines are as smart as Crafty, however. Micro-Max would seach to d=100, which would still take only an instant if mate-in-3 is looming, since it doesn mate-distance pruning, and all d>5 searches will be as fast as the d=5 search. But there are engines that do not even do mate-distance pruning, and they search for minutes even on mate-in-1 positions.

Yup, I agree that not all engines will complete early. My point is that Winboard engines don't have to wait for an external command to stop searching.

[CThinker]

Like many other things, it is a big deal for me and for almost nobody else on the planet.

I often will take a list of 150 positions, and let a very strong engine analyze at (for instance) 1/2 hour per position over the weekend. It will often be the case that ten hours or so is completely wasted because of sitting there doing nothing for 20 of the positions.

It is especially true when analyzing a large segment of endgame positions.

It probably bothers nobody else at all, but it bugs me to no end.

It seems to me that you bring this upon yourself. If you want to do not exactly 1/2 hour per position, but 1/2 hour maximum, you should use the engine not in analysis, but in fixed-max-time-per-move mode (st in WB, movetime in UCI). Then the engine will terminate the search if there is nothing left to search.

I think Dann is being misunderstood here.

Here is an example of a Winboard sequence:
setboard POSITION1
tc 6000 // 1 minute
go
setboard POSITION2
tc 6000 // 1 minute
go
setboard POSITION3
tc 6000 // 1 minute
go

That entire process will take at most 3 minutes.

However, if POSITION1 and POSITON2 happens to be mates and can be solved in 3 seconds each, then the entire process will just take 1 minute and 6 seconds.

But with UCI, the entire process will always take 3 minutes, no matter what the positions are.

Slight correction:
The search time command should not be "tc". It should be "st".

[Gian-Carlo Pascutto]

The problem is that, with UCI, the engine will use of all of the time control even if the solution is reached in less time (like in a mate).

That has nothing to do with UCI, just look at the spec:

* ponder
start searching in pondering mode.
Do not exit the search in ponder mode, even if it's mate!
....
* movetime
search exactly x mseconds
...
* infinite
search until the "stop" command. Do not exit the search without being told so in this mode!

Note that there is NO remark in movetime about exiting earlier. Granted, the spec could have been clearer, but to me this implies exiting earlier is allowed. You cannot break anything by doing this, because the GUI cannot expect you to obey the duration exactly anyway (how would you define the duration? wallclock/cpuclock? You cannot exit halfway through a node, etc). This is very different from obeying an "infinite" duration (ponder/infinite).

Deep Sjeng does not behave like you describe and will exit when the mate is found and no shorter one is likely to be found.

[Gian-Carlo Pascutto]

No it is not "baseless". If you ignore a token, you take a step down a slippery slope since the next token might depend on the context of the previous one.

Extending the protocol in that way (altering the expected parsing of an understood command by making it context dependent on a not understood one) would be terminally stupid yes. Which is why nobody does that.

One doesn't "ignore" flags in TCP/IP. So I am not sure what that is about. You don't have to use any/all of the flags if you don't want to. Unless you are talking about the real "reserved" flags that have _no_ explicit meaning. Not sure how that applies to this circumstance...

What do you think a TCP stack should do if one of the reserved flags is set? Drop the packet? Or ignore the flag?

[Roger Brown]

[quote="Gian-Carlo Pascutto


[SNIP]

Deep Sjeng does not behave like you describe and will exit when the mate is found and no shorter one is likely to be found.[/quote]



Hello Gian-Carlo,

All the best for the season personally and professionally and with Deep Sjeng.

I have a question. I am becoming more interested in the coding side of things and as a patzer chessplayer and idiot programmer, isn't the spec you are ignoring precisely due to your genius as a programmer?

I mean, you are programming your engine to behave rationally and quit on finding mate but the spec you cited clearly expects the engine to consider the position until the time period is ended regardless of whether there is anything further to consider.

I understood duration to be sent by the gui which keeps track of the time and other matters of the game state for UCI engines, does it not?

In UCI mode, how does Deep Sjeng tell the gui that it is quitting now as the search is complete?

Forgive the (possibly) basic, idiotic and strange questions.

Later.

[Sven]

Hi Roger,

In UCI mode, how does Deep Sjeng tell the gui that it is quitting now as the search is complete?

I would expect that his engine terminates the search with the normal "bestmove" command sent to the GUI.

[...] isn't the spec you are ignoring precisely due to your genius as a programmer?

I mean, you are programming your engine to behave rationally and quit on finding mate but the spec you cited clearly expects the engine to consider the position until the time period is ended regardless of whether there is anything further to consider.

As to "ignoring the spec", Gian-Carlo clearly stated (and this is also my opinion) that the UCI spec is not precise enough in this case. It explains the "movetime" subcommand with "search exactly x mseconds" but does not mention the obvious exceptional case of exiting earlier, while it does so for the "ponder" and "infinite" subcommands.

The usual way to deal with a spec that is found to be imprecise to some degree is to fix it. However, we cannot expect the UCI spec to be changed. This situation immediately gives room for interpretation. In the given case Gian-Carlo has provided his understanding, and I think it is quite reasonable. I can also tell you why:

You can advise a modern chess engine to search "exactly 60000 mseconds" from a mate-in-6 position. Nevertheless it will take only very short time to find the mate, after only very few iterations of search. Now there are these choices, after having found the mate:

a) Continue "searching" until the predefined time is up. Run more iterations, with a "virtual" maximum search depth increasing consistently, but always finding the same shorter mate variation instantly without even going to the full maximum depth. Pretty useless kind of burning your CPU.

b) Stop searching internally but do not report the final result yet, instead fall asleep until the predefined time is up, then wake up and report your mate. Would you expect that?

c) Stop searching internally like b) but instead of falling asleep, play a nice video until the time is up. I guess you would prefer this one

d) Go the rational way by exiting immediately when finding the mate. I'd prefer it this way, regardless of the protocol being used.

For me the most convincing point of Gian-Carlo is that doing so does not break anything.

Sven

[hgm]

Yup, I agree that not all engines will complete early. My point is that Winboard engines don't have to wait for an external command to stop searching.

Neither do UCI engines. They are told before each move how long they can think, (per movetime, wtime+btime+movestogo command), and will spontaneously produce a move when they think they have used enough time.

[Roger Brown]

Hi Roger,

Hello Sven!

Alright, so it is good to hear from you.

c) Stop searching internally like b) but instead of falling asleep, play a nice video until the time is up. I guess you would prefer this one

YES.

d) Go the rational way by exiting immediately when finding the mate. I'd prefer it this way, regardless of the protocol being used.

For me the most convincing point of Gian-Carlo is that doing so does not break anything.

Thank you for your usual clear explanations, comprehended by even me. I am afraid that I did not know that a UCI engine had the requisite ability to spontaneously stop and send the result as bestmove to the gui.

Sigh.

Later.