Asymetric Node/move in winboard for UCI engines

[hgm]

Do you really think i didn't know all stuff ? I didn't say how i use a dump or in which stages i do it exactly. So, please don't tell me how you do it or in which cases. If i have a problem i would ask for it.

Best,
Daniel

ps: It seems to be that all your postings going this way und i guess you just wanna help. Thanks for that, but i don't need it.

Well, based on your remarks, it seems you do. In my initial post exlaining the nps command on your request, I explicitly gave deterministm (and thus reproduceability of errors) as one of the applications of this feature, which is also what Miguel streses, and you still deny that it could be useful for this...

And keep in mind that there are other readers as well. even if you yourself realize that what you write is nonsense, others might not realize it without me pointing it out...

So your suggestion is that programmers should start to hide the real NPS like Rybka does ? Also Rbka's NPS are changing in tiny steps during a game.

That is indeed exactly what I am suggesting. At least when running in WB nps mode. If you would read the protocol specs, you would see in the description of the nps command that I explicitly invite authors to do just that.

[hgm]

[edit] Well, I guess the latter is not true anymore, I apparently edited this out of the protocol specs. But the following remarks could still be made:

* If all engines speed up their nps in the end-game, this mostly neutralizes the effect of using nodes in stead of time as a basis for how deep to seach.
* So if this bothers the author (and there aree many applications where it does not hurt), thee would only be a need to change the counting method when the efect would be exceptionally large or small without such change.

In Joker I count QS nodes only half, as I empirically found tht QS nodes were processed about twice as fast as full-width nodes. This makes the nps of Joker pretty constant over the game phases.

[bob]

How do I do that? My nps varies by a factor of 2-3x from opening to endgame. Which NPS do I pick? And how do I prevent the distortion I just mentioned? Try to pick opponents that have exactly the same NPS changes over the game? Not so easy.

If I were you, I would try to fix the method of node counting in my own engine so that it would no longer have this variation. There must be a reason for this variation. You mention a very expensive evaluation term, used before castling. Well, if 85% of your time use is evaulation, count evaluations, and count those that contain that term double. If tablebase probes take 5 times as long as evaluations, add the probes to the count and count them for 5.

So now we have to redefine the classic definition of the term "node" as well? A node is precisely defined. Must programs search faster in the endgame since there are fewer pieces and move generation is one cost incurred at almost every node. There are lots of reasons the node speed varies. Evaluation cost is amortized across all nodes in a path. Move generation costs vary. Selecting moves can have variable cost. In short, this is simply a dynamic value, otherwise no one would print it out, if it were truly a constant that doesn't change, as it would be pointless to always say "nps=19.2M" for Crafty. But it isn't a constant.

What you are wanting is for the program to somehow estimate some unit of computation, which is unrelated to the term "node". And use this so that you can tell it how many of those "units" to use for a search. This seems to be a step in the wrong direction when this idea includes the term "nodes". I can't "adjust" the number of nodes I search. I leave that kind of Crap to Rybka & clones.

WB protocol does not specify exactly what the node count means, and it would be impossible to do this anyway, as it is dependent on implementation details. So there is no need to report a true count of nodes in the tree. Micro-Max counts IID iterations in stead, because it does move generation anew for every iteration, and spends most of its time on that. And as it does IID in every node, it means that each internl node will be counted as many. Who cares? As long as you report the same count in the Thinking Output as what you base your timing decisions on, node-based play under WinBoard will work.

I had guessed that was where you were heading. And there is nothing wrong with the idea, so long as everything can be quantified. But it is not that easy, IMHO. Things that can change NPS:

(1) move generation time. More pieces, longer to generate 'em.
(2) evaluation time. More pieces, longer to evaluate. Some pieces of the eval might only be hit in certain types of positions (passed pawns, opening development, drawish position analysis (limited material left). Etc.
(3) Search tree itself. Trees with lots of checks are slower for me, because there are fewer legal moves out of check, than there are legal moves in a normal position. Positions near the 50-move boundary are slower because the repetition list gets quite long. And the list goes on and on.

This would be a project unto itself to try to come up with a "unit" of computation that is consistent from opening to endgame and everywhere in between. At least for my program. Because I have spent so much time making sure that I bail out on something as soon as possible, rather than doing unnecessary loops, etc.

And if you cannot or do not want to do that, than this method is simply not for you, because your engine has too variable a node rate. But that doesn't mean it is not worth having, as others have engines that do not have this problem, or use the facility for purposes where they do not care about this problem. Unlike Crafty, WinBoard is not a "Bob-only" project...

Of course not, but it does need to be carefully explained, with a "surgeon general's warning" afixed, so that no one uses this for testing and gets badly burned in doing so because of the above very unexpected issues that most would not even think about, until after they had made a ton of mistakes about which changes to keep and which changes to throw out.

[bob]

[edit] Well, I guess the latter is not true anymore, I apparently edited this out of the protocol specs. But the following remarks could still be made:

* If all engines speed up their nps in the end-game, this mostly neutralizes the effect of using nodes in stead of time as a basis for how deep to seach.
* So if this bothers the author (and there aree many applications where it does not hurt), thee would only be a need to change the counting method when the efect would be exceptionally large or small without such change.

In Joker I count QS nodes only half, as I empirically found tht QS nodes were processed about twice as fast as full-width nodes. This makes the nps of Joker pretty constant over the game phases.

I remember the day I bought my future wife her engagement ring in Mobile AL. A band was playing in the park, where we went to listen while waiting on the jeweler to mount the diamond and size the thing. They played the song "Judy in Disguise."

Why did I mention that? This sounds like "Rybka in Disguise". Nodes really are precisely defined. For most of us. Rybka excluded of course.

[hgm]

Well, I am not particulrly interested in the exact size of the tree according to the classical definition. There is really nothing I could do with this information. For me the nodes per second is just a diagnostic to see if my engine is healthy, and gets the CPU attention it should have. I don't want to use it as an indiation to see if the three is opening-like or end-game-like For that I can look on the board.

But that is just digressing. I think you attach way too much value to the time distribution over the game. Logically, this cannot be a problem at all:

* If there is a universal speedup in nps towards the end-game, your own engine will show it as much as other engines, and the effect will largely cancel.
* If engines behave differently over a wide range of speedups, you simply pick a few that have a similar speedup as your own.

[Daniel Mehrmann]

Well, based on your remarks, it seems you do.

Which remarks ?? I never wrote something how i use my dumptree stuff.

So your suggestion is that programmers should start to hide the real NPS like Rybka does ? Also Rbka's NPS are changing in tiny steps during a game.

That is indeed exactly what I am suggesting. At least when running in WB nps mode. If you would read the protocol specs, you would see in the description of the nps command that I explicitly invite authors to do just that.

Okay, thanks. I have no further questions!

Best,
Daniel

[hgm]

Which remarks ?? I never wrote something how i use my dumptree stuff.

That is exactly the point. Your remarks are completely off topic. We are discussing here how playing by nodes in stead of by time helps during program development, because the resulting determinism guarantees that any errors you encounter will by reproducible, rather than illusive. As I initially said, this is one of the reesons I provide it, and it is one of the reasons Miguel mentions for liking it.

Then you come with your remark 'not in professional testing' and start babbling on that it is better to make tree dumps. That is just totally beside the point, because it addresses what you do _after_ you found an error, while we are discussing _how to find_ the error. So this suggests you have no idea what we are actually talking about. It is also not clear what making a tree dump has to do with using nodes rather than time to limit the tree size; If there is any place where it is totally immaterial if you use one or the other, it should be there (quite unlike the case of testing for program improvements that Bob addresses.)

So are you just trolling us, or what???

[Daniel Mehrmann]

So just a reminder: NPS->node->bugfixing->bob does->alternative idea dumptree. This is clear in topic. You can use NPS for testing (A vs A, A vs B...) as _you_ wrote. So you can use it to _find_ an error and _after_ an error. Just repeating your own words. Why do you simply lying ?

So are you just trolling us, or what???

Nobody called me a troll in over 15 years computerchess history so far.
I don't need this talk level you're starting to go on. This will be my last answer.

[hgm]

Is anyone supposed to understand any of this? To me it is total gibberish...

So let me just go back to square one and reiterate what I was saying in the first place:

1) Using an internal time standard such as nodes eliminates communication delays, which is very important in ultra-fast games
2) On computers that are not dedicated to Chess, using an internal time standard allows you to do engine testsing as a background process
3) Elimination of time jitter gives you determinism, which makes bugs amnifest themselves reproducibly, which is very helpful in debugging.

For (1) and (2) you have to take care that the variation in node rate (whatever they report as " nodes") of the engines is sufficiently similar, or you _might_ get distorted results. For (3) it must not be an SMP engine, where timing jitter between the search threads causes intrinsic time jitter, which cannot be eliminated this way.

But that leaves enough cases to make node-based time controls a very useful protocol addition. Plus that an addition of this or a similar function was called for anyway, to make it possible to use the go nodes functionality of UCI engines when running under Polyglot.

Now if someone has to remark something _relevant_ and _sensible_ about this, it is of course more than welcome.

[michiguel]

Well, i think Bob wrote a lot of cons arguments already. So, there is no need to write more. It doesn't look like you'll accept any point of view.

And how about the pros?

For professional testing (search/eval) ? None! It's just gambling.

It might be interesting for bugfixing as Bob pointed out already. But for "Node"-Bugfixing it's much better to dump the searchtree and analysis it! That's the way i do handle these kind of problems.

If you cannot reproduce the bug, what tree are you going to dump?
The most critical part of debugging, is reproducing the bug. Then, there are many tools you can use. Dumping the tree is one of them. I want to do both, not one or the other.

Do you ever try to dump your searchtree and use your own tool for graphic analysis ?

I do that and more, but it is irrelevant to the discussion.

I had in my old to do list one way to force reproducibility => log at what node exactly the search stopped. After that, I should be able to write a tool to force the game to go through the same path. However, this addition in the protocol will provide a general solution and will make things easier for everybody that is interested in reproducing potential bugs.

This is really good for those type of bugs that manifest themselves 1 in 1000 games and become incredibly difficult to catch.

Miguel

Best,
Daniel