fixed nodes testing

[Don]

Yes, but remember in UCI the interface sends this information and no interface that I know of knows how to pretend that nodes is time and do the accounting. Have you ever heard of a 40 million nodes in 2 minute time control? No UCI support so you must build it into your tester and also provide a way for the engine to report precisely how many nodes were consumed to the interface and also node forfeits would have be become a reality.

Well, that is what you get for using an inferior protocol. WinBoard / XBoard would support all that, throught the options -firstNPS, -secondNPS.

I think the real issue is not the total number of commands that a protocol supports but whether the underlying abstraction is sound. I think time has shown that both protocols are sound.

UCI is a stateless protocol and assumes that the engine knows nothing about the state - it is provided by the interface. That makes things simple for the engine implementer and is either an advantage, or a disadvantage depending on your point of view. It's a more modern concept (modular design) but that does not imply that is't always the best thing to do.

Either protocol could benefit in functionality by adding endless numbers of additional "commands" but I personally believe the soundest protocols should be relatively minimalistic and provide the low level functionality to enable third party tools to be easily built that can do most everything one could want.

So in your example the firstNPS functionality could be added to UCI without changing anything - I don't think it reflects on the superiority or inferiority of any particular protocol.

[Desperado]

I don't think there is any need to specify a nodeCount resolution. Unlike reading the time, testing the number of nodes is extremely fast. So it should simply compare the node count to the node limits (converted from the time limits in msec by multiplying the latter by the NPS parameter divided by 1000) in every node.

Maybe you are right, that is why i difference between _basic_ and _advanced_ implementation in one of my posts. Time will show
if _ultra fast_ games in such a nodecount mode need such a resolution,
and are worth to implement.

My point of view currently is that. It would be much more motivating to work on such a topic if not only _one_ engine benefits. It would be
nice to see that various programmers can use (and support) such an _uci tuning kit_. The uci-option list would be a good interface imho.
Existing testers like LittleBlitzer or cutechess-cli can be used furthermore.

if there is a bad conscience by landing in the mentioned gray area of uci-handling, i suggest those people that they throw it out in public release versions.
And people who can live with it, use a _developer_ version can do so.

So finally all are happy

Michael

[Don]

Just some comments, before i leave...

* uci will not be broken.
- i was talking of _options_ like hash is an option every
uci engine supports (not a new unsupported command)

You miss the point. What would be broken is the stateless abstraction that UCI provides as a convenience to the programmer.

Tell me how you would implement a node level that stopped early on simple moves and use the nodes saved for future searches. You must do this without knowing anything about previous states, such as how many nodes you used on the last move.

- the tester (gui) only has to send the infinite command. Then
we will end up in gray area like what to do when your iteration limit
is reached already. Practically _all_ engines are sending the bestmove
in infinity mode, when search is done. So you can easily send the
bestmove at each iteration, nodecount , timebased or whatever
controlled mode when receiving the infinite command.

- so because there are no _new standards_ and you read about the
_improved_ idea i was describing in the next post, you will easily
see that there cannot be any polling problems at all. Everything
depends on the support of the engine developer how he will implement
nodecount handling (like you already asked for exact-nodecount-handling).
If the develpers who are interested in such
_options_ they will implement it. Uci protocol is kept untouched.

cheers

Michael

[hgm]

Hmm... Interesting philosophy. My car is not broken, because it can be fixed. But in the mean time I am standing along the road without spark plugs.

I guess you could reason away every difference between anything that way. But in my book UCI as it exists has the severe drawback that it does not have wnodes and bnodes options on the go command as counterparts to wtime and btime. Only nodes as counterpart to movetime. That makes it impossible for the tester client to make the engine do things that can be done with WB protocol by XBoard.

Of course it can be fixed. But to fix it without making a new dialect of UCI, I still think the best and easiest way would be to have the tester client send

position startpos moves e2e4 e7e5 g1f3
setoption name NodesPerSecond value 500
go movestogo 39 wtime 59000 btime 59500

which pre-emptively causes interpretation of the wtime and btime options as 29500 white nodes (59 * 500) and 29750 black nodes (59.5 * 500).

[Desperado]

I don't think there is any need to specify a nodeCount resolution. Unlike reading the time, testing the number of nodes is extremely fast. So it should simply compare the node count to the node limits (converted from the time limits in msec by multiplying the latter by the NPS parameter divided by 1000) in every node.

I don't see the point of sending go infinite in stead of sending movestogo, wtime and btime. It would require a modification in the tester client anyway to send go infinite in a mode where it expects to get a bestmove back, so you might as well change the way the clock is decremented, and then send that time in a normalgo command.

every uci interface will try to execute the bestmove it receives. Also in infinite mode. (so far as my experience goes ). every uci interface will flag time in any other time mode. In simple words, there is no modification
necessary.

of the other time based commands is used.

[Desperado]

Just some comments, before i leave...

* uci will not be broken.
- i was talking of _options_ like hash is an option every
uci engine supports (not a new unsupported command)

You miss the point. What would be broken is the stateless abstraction that UCI provides as a convenience to the programmer.

Tell me how you would implement a node level that stopped early on simple moves and use the nodes saved for future searches. You must do this without knowing anything about previous states, such as how many nodes you used on the last move.

- the tester (gui) only has to send the infinite command. Then
we will end up in gray area like what to do when your iteration limit
is reached already. Practically _all_ engines are sending the bestmove
in infinity mode, when search is done. So you can easily send the
bestmove at each iteration, nodecount , timebased or whatever
controlled mode when receiving the infinite command.

- so because there are no _new standards_ and you read about the
_improved_ idea i was describing in the next post, you will easily
see that there cannot be any polling problems at all. Everything
depends on the support of the engine developer how he will implement
nodecount handling (like you already asked for exact-nodecount-handling).
If the develpers who are interested in such
_options_ they will implement it. Uci protocol is kept untouched.

cheers

Michael

1:
===

i dont have to save anything. Maybe you reread my second (think it is the second post with _improved_ proposal) post.

2:
===

a:

You are asking for refinement of nodecount handling, which is 100% limited to inner handling of the engine. So my proposal was exactly doing
this. no modifications beside adding _options_.

b:

receiving an infinite command but sending bestmove without stop is
a lack in uci, and was all the time. This can be perfectly used, i agree
is gray area with respect how uci should be used.

But you should differ between test bed and some kind final version of an engine.

Filling up this lack with some kind of _tuning options_ , all uci engine developers can use, makes 100% sense to me, in any gui or tester which
supports the uci protocoll.

So maybe there is something wrong with my nodecount handling proposal
itself. No problem, let us continue and research better ways.

But to have a problem to use a lack in the protocol itself, which will break
nothing, and you use mainly in your test bed, and you can throw it out whenever you like, and and and is no argument.

Michael

[Desperado]

Ok, i want to add that without saving game state controllers some things will change of course. But arent we looking
for a solution that comes close as possible (but are not 1:1 translated) to time based controlled games, using nodecount as controller ?
There will never be a way to get a 100% translation from time based controled modes to nodecount controled modes.

[hgm]

every uci interface will try to execute the bestmove it receives. Also in infinite mode. (so far as my experience goes ). every uci interface will flag time in any other time mode. In simple words, there is no modification
necessary.

1) I don't think this is true, and
2) You can never claim such things when the protocol does not prescribe it, because you can't know what interfaces will be written in the future.

Engines send a bestmove after a ponder miss, and it should not be executed.
Engines send a bestmove when the engine wants to feed them a new position during analysis, and it should not be executed.

[hgm]

Tell me how you would implement a node level that stopped early on simple moves and use the nodes saved for future searches. You must do this without knowing anything about previous states, such as how many nodes you used on the last move.

I don' t understand this remark. Compared it to this one, obtained by a substitution nodes -> time:

Tell me how you would implement a time level that stopped early on simple moves and use the time saved for future searches. You must do this without knowing anything about previous states, such as how much time you used on the last move.

Do we agree that the latter can be easily told, because it is exactly what all existing interfaces do?

[Don]

Tell me how you would implement a node level that stopped early on simple moves and use the nodes saved for future searches. You must do this without knowing anything about previous states, such as how many nodes you used on the last move.

I don' t understand this remark. Compared it to this one, obtained by a substitution nodes -> time:

Tell me how you would implement a time level that stopped early on simple moves and use the time saved for future searches. You must do this without knowing anything about previous states, such as how much time you used on the last move.

Do we agree that the latter can be easily told, because it is exactly what all existing interfaces do?

I don't think I'm making myself very clear. The proposal was to implement "go nodes x" in such a way that the program would try to complete an iteration or not start a new one if the number of nodes remaining was too small. In order to do that the engine would have to be able to track how many extra nodes it builds up from move to move. That breaks the UCI abstraction. It doesn't make it impossible or even difficult, but it's an ugly hack. Another way to implement it is to ignore state and just not start an iteration unless you have 40% of the time remaining, and never stop an iteration until it completes. That would be unacceptable in my opinion as that would make some games last much longer than others. You might get 10 moves in a row where you played very quickly for example or 10 moves where you think twice as many nodes.

The correct way to do this is to add a command to UCI which provides support and which the user interface would also have to be aware of. Something like the -firstNPS and -secondNPS would be a good way to implement it in UCI.

By the way, I completely disagree that the value of a protocol is simply the number of commands it has. That is hacker mentality, not solid engineering. So you believe that if we added all the commands that WB has plus 2 or 3 move it would make it a better protocol than WB? I don't really believe you would agree to that (even though I probably would.)


I'm thinking of building a new protocol. It will be based on XML with JSON embedded inside the XML and nested or recusive PV functionality, etc and it will have every functionality the WB and UCI has and even more. Do you actually believe that would make it a "superior" protocol? I forgot about the part that the communication stream uses an error correcting check-sum protocol and options for encrypting the information flow going back and forth. Surely this far superior to all those other protocols that cannot do this.