polyglot not reporting mate scores to winboard

Discussion of chess software programming and technical issues.

Moderator: Ras

mar
Posts: 2665
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: polyglot not reporting mate scores to winboard

Post by mar »

hgm wrote:Indeed, the main problem is not in the protocol, but in the engines. Every WB engine is using its own mate score, some counting DTM in ply, others in moves. Most interfaces only understand numeric scores. WinBoard itself, for instance, will completely ignore Thinking Output that has a non-digit in its score field. But there exist enough different numbers to requisition some of them to indicate mate scores,and this would provide backward compatibility with the interfaces (i.e. engines using the new standard could still run on old interfaces). Of course the old interfaces would not recognize them as mate scores, but give them the normal treatment that all scores get. But having them display "1000.06" for mate in 6 does not seem that much worse than having them display "Mate6". The "1000" is as recognizable as "Mate".

It would only take me a couple of minutes to change WinBoard such that it would print "M6" in stead of 1000.06, wherever it prints scores. And perhaps half an hour to figure out where in Polyglot I have to interfere to make it print 100006 in stead of 9989. But now I am going to bed! :wink:
I understand. Please don't get me wrong I have nothing against the protocol itself. Even if I prefer UCI it's just a matter of taste. I thought that it handles mate scores but it doesn't: my mistake. I have to live with that. Good night and thanks for answer :)
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: polyglot not reporting mate scores to winboard

Post by Sven »

hgm wrote:I think Sven and I have a fundamentally different view on this. In mate scores, the highest number usually indicates the most distant mate (M3 is better than M6).
"M3" and "M6" are no mate scores but string representations of these. The scores are numbers, and if M3 is better than M6 (which is obviously true) then the number represented by "M3" should be greater than the number representing "M6". Otherwise you request a mapping by the engine that maps its internal (MAX_VALUE - 3) score to your (100000 + 3) and (MAX_VALUE - 6) to (100000 + 6). But why would you want to violate the KISS principle by doing that?
hgm wrote:And in my opinion we must never use a score in the 16-bit range, exactly because many engines internally use 16-bit scoring.
Engines with internal 16-bit scoring will also have their special range for mate scores within the 16-bit bounds. Show me any engine that has 16-bit scores and allows "centipawn evaluation" values with an absolute value greater than 32500. 9 queens, 2 rooks/bishops/knights and an astronomical huge positional bonus of, say, 5000 cp (~ 4-5 queens or more, usually a bad idea for positional scoring) will never make it even above 20000 cp.

"Millipawn scoring" definitely requires 32-bit values, so your argument should have been in favor of these :-)

But you can try to make it comfortable. Offer two options "MateScoreBase" and "MateScorePolicy" that can be set for engines, where the former is the MAX_VALUE used as a base for all mate scores (e.g. 30000, 100000) and the latter is an enum value allowing to set whatever you want to allow as a method for mate score interpretation. The task of the GUI would be to translate numbers sent by the engine into strings like "M6" ("+M6"?). Use reasonable defaults for both (you know my vote ;-) ), and make the mechanism available for both WB and UCI engines. You might even say that the most reasonable default would be not to do any translation of mate scores at all, for compatibility reasons.

What do you think about it?

Sven
User avatar
hgm
Posts: 28390
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: polyglot not reporting mate scores to winboard

Post by hgm »

Sven Schüle wrote:Otherwise you request a mapping by the engine that maps its internal (MAX_VALUE - 3) score to your (100000 + 3) and (MAX_VALUE - 6) to (100000 + 6).
Well, such mapping is intrinsic in the entire issue of mate scores, right? There is no engine in the world that uses ascii strings for scores in its search. So if you want engines that print non-numeric strings for mate scores, it will have to recognize their own numeric mate scores, and convert them to a string. Not a constant string, but a string that somehow reflects the DTM, which they have to deduce from their numeric internal score. Mapping internal numeric scores on text strings in a non-trivial way (i.e. other than printing the numeric value) is unavoidable.
But why would you want to violate the KISS principle by doing that?
I don't see how this applies. Why do you consider printf("Mate%d", dtm)any simpler than printf("%d", 100000+dtm) ? It would certainly be far more complex on the GUI end, where we would now have to read the score field as a string, throw a strncmp on it to test if it happens to start with "Mate" or "Mated" or "-Mate", and in that case convert it to some number (perhaps by adding 100000 to the reported dtm?) as the GUI does not handle scores internally as text strings either. All for naught...

Super-simple it already is now. People that prefer to represent mate in 6 by 9994 already get exactly what they want. But people complain,it is not what they want...
But you can try to make it comfortable. Offer two options "MateScoreBase" and "MateScorePolicy" that can be set for engines, where the former is the MAX_VALUE used as a base for all mate scores (e.g. 30000, 100000) and the latter is an enum value allowing to set whatever you want to allow as a method for mate score interpretation. The task of the GUI would be to translate numbers sent by the engine into strings like "M6" ("+M6"?). Use reasonable defaults for both (you know my vote ;-) ), and make the mechanism available for both WB and UCI engines. You might even say that the most reasonable default would be not to do any translation of mate scores at all, for compatibility reasons.

What do you think about it?
What happened to the KISS idea? :?

I think it is bad policy to allow everyone to do exactly what he wants, and cater to all of that nonsense in the GUI. We might as well abandon the concept of protocols alltogether, and say that any engine can use any set of commands and responses it feels like, and then put the burdon on the GUI to recognize the engine and know what it has to send in stead of 'new' or 'go' to make it work. Or let the GUI edit engine INI files of WB engines in response to user changes in the Engine #1 Settings dialog... Or let the user configure the GUI so that it knows how to configure the engine in any of a zillion ways, so that the user does not have to configure the engine in stead.

That is not how we do things. We define a protocol where the same thing always has to be done the same way.

I already consider it madness to allow engines to define non-standard protocol through standard protocol. In the past there has been a proposal to let engines tell the GUI through a feature command what their mate score is. That serves absolutely no purpose. It would not work for any existing engine because they do not send the feature, and if engines haveto bemodified, they might as well be modified to use a mate score we proclaim standard in stead of sending a feature command we proclaim standard...
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: polyglot not reporting mate scores to winboard

Post by Sven »

hgm wrote:
Sven Schüle wrote:Otherwise you request a mapping by the engine that maps its internal (MAX_VALUE - 3) score to your (100000 + 3) and (MAX_VALUE - 6) to (100000 + 6).
Well, such mapping is intrinsic in the entire issue of mate scores, right? There is no engine in the world that uses ascii strings for scores in its search. So if you want engines that print non-numeric strings for mate scores, it will have to recognize their own numeric mate scores, and convert them to a string. Not a constant string, but a string that somehow reflects the DTM, which they have to deduce from their numeric internal score. Mapping internal numeric scores on text strings in a non-trivial way (i.e. other than printing the numeric value) is unavoidable.
But why would you want to violate the KISS principle by doing that?
I don't see how this applies. Why do you consider printf("Mate%d", dtm)any simpler than printf("%d", 100000+dtm) ? It would certainly be far more complex on the GUI end, where we would now have to read the score field as a string, throw a strncmp on it to test if it happens to start with "Mate" or "Mated" or "-Mate", and in that case convert it to some number (perhaps by adding 100000 to the reported dtm?) as the GUI does not handle scores internally as text strings either. All for naught...

Super-simple it already is now. People that prefer to represent mate in 6 by 9994 already get exactly what they want. But people complain,it is not what they want...
But you can try to make it comfortable. Offer two options "MateScoreBase" and "MateScorePolicy" that can be set for engines, where the former is the MAX_VALUE used as a base for all mate scores (e.g. 30000, 100000) and the latter is an enum value allowing to set whatever you want to allow as a method for mate score interpretation. The task of the GUI would be to translate numbers sent by the engine into strings like "M6" ("+M6"?). Use reasonable defaults for both (you know my vote ;-) ), and make the mechanism available for both WB and UCI engines. You might even say that the most reasonable default would be not to do any translation of mate scores at all, for compatibility reasons.

What do you think about it?
What happened to the KISS idea? :?

I think it is bad policy to allow everyone to do exactly what he wants, and cater to all of that nonsense in the GUI. We might as well abandon the concept of protocols alltogether, and say that any engine can use any set of commands and responses it feels like, and then put the burdon on the GUI to recognize the engine and know what it has to send in stead of 'new' or 'go' to make it work. Or let the GUI edit engine INI files of WB engines in response to user changes in the Engine #1 Settings dialog... Or let the user configure the GUI so that it knows how to configure the engine in any of a zillion ways, so that the user does not have to configure the engine in stead.

That is not how we do things. We define a protocol where the same thing always has to be done the same way.

I already consider it madness to allow engines to define non-standard protocol through standard protocol. In the past there has been a proposal to let engines tell the GUI through a feature command what their mate score is. That serves absolutely no purpose. It would not work for any existing engine because they do not send the feature, and if engines haveto bemodified, they might as well be modified to use a mate score we proclaim standard in stead of sending a feature command we proclaim standard...
Good morning Harm-Geert!

You are far away from what I wrote, by about 100% :-)

Engines send PVs containing numbers for the score. I never wrote engines should send "Mate6" or "M6" instead. The numbers contained in a PV line sent by the engine are usually direct results from the search, so a higher value is a better value (except for engines that do scoring from white viewpoint, which I ignore here).

If an engine finds a forced mate for itself then its search returns a value calculated like "MATE_VALUE - something", where "something" is usually a positive number representing distance to mate, either counted in plies or in full moves. The search does not return "MATE_VALUE + something" since that would make the search favor longer mates over shorter ones based on minimax (negamax) principle.

Now the task of the GUI could be to translate "MATE_VALUE - something" into a string like "M6" if something==6 (EDIT: and counting is in full moves, otherwise it would be something==11). That would be a trivial operation if there is a precise formula for that translation. The implementation itself is straightforward.

And that is all I was writing about. I do not understand what you are making out of it. If the topic was a different one then I'm sorry for interrupting, I thought it was clearly about that task, though.

As to your unwillingness to offer new options, features, or whatever variability for that mate scoring task, I can understand your intentions but fact is that the huge variety of formats engines are using today to represent their mate scores is caused, at least partially, by not having standardized that part on the protocol side - WB as well as UCI. Which is something you are _not_ to be blamed for, of course, it is simply a historical fact that is to be accepted.

And if there is desire to change that situation, even after 10, 20, 30 years, then you always have to take into account the hundreds of engines and engine versions that will continue to exist (and to be included in tests all over the world) for many years, without ever being changed to keep up with new protocol standards. So you _must_ remain compatible with changes. And that is why offering a way how engines (WB or UCI) can tell the GUI how their mate scores shall be interpreted could be a way to go, if that is technically possible.

As to your last sentence above, neither a feature nor an option describing the mate scoring format of an engine must be proclaimed as "standard". It is optional, as the words "feature" or "option" say, so there must be reasonable defaults for it. In favor of existing and unchangeable engines you would certainly set the default policy (if no option or feature is set) to "no translation, GUI shows raw number as sent by engine". Note, I am talking about configuring the GUI, not the engine. For WB I think this is best done with a "feature", I don't know how to do it for UCI engines, nor if it can be done at all.

Sven
User avatar
hgm
Posts: 28390
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: polyglot not reporting mate scores to winboard

Post by hgm »

UCI engines are expected to send real mate scores, not? Like score mate 6 in stead of score cp 31 for a non mate score.

Letting the user configure the GUI for transforming scores on a per-engine bases before printing is something that can always be done. But it has nothing to do with the protocol, it is a GUI implementation issue.

Still, I think it is an undesirable way to go, because it puts the burden on the user. I prefer to use solutions that work automatically. Even a solution as standardizing the mate score to a certain numeric value N-dtm would require most engines to recognize mate scores they use internally, and convert them to the standard value, even if this would be just by adding a constant. E.g.

Code: Select all

if(score > MATESCORE-100) printf("%d", score-MATESCORE+N); 
else printf("%d", score);
and often they would have to correct for the difference of counting in ply or moves, and you would get something like

Code: Select all

if(score > MATESCORE-100) printf("%d", (score-MATESCORE)/2+N); 
else printf("%d", score);
I don't think an extra minus sign could be said to significantly add complexity:

Code: Select all

if(score > MATESCORE-100) printf("%d", -(score-MATESCORE)/2+N); 
else printf("%d", score);
And as it has the advantage that the dtm will be easily recognizable even in GUIs that do not attach any special significance to scores in this range, I think that minus sign is well worth it. If you consider that (WB) engine authors often go as fast to emit the pv as SAN just to make it look prettier in the engine-output window, this is really a discussion about nothing.

(Of course WinBoard can even display SAN for engines that send long-algebraic now, if the user prefers that format, through the /fSAN and /sSAN options.)
Aleks Peshkov
Posts: 926
Joined: Sun Nov 19, 2006 9:16 pm
Location: Russia
Full name: Aleks Peshkov

Re: polyglot not reporting mate scores to winboard

Post by Aleks Peshkov »

PGN standard was written years ago and it defined how exactly mate score should correspond to centipawns (in EPD section). Arena GUI understands this mate values from ancient times.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: polyglot not reporting mate scores to winboard

Post by Evert »

Why not have a new command that allows the engine to announce "mate in N" (or maybe "{Mate in N}")? There's then no problem with engines that don't implement that command (they'll simply never announce mate) and there's no special handing/interpretation of magic scores.
Sounds like a win-win solution to me...
User avatar
hgm
Posts: 28390
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: polyglot not reporting mate scores to winboard

Post by hgm »

I don't see what exactly you 'win'. If we would define 100000000N to be the announcement for mate in N, engines that do not implement it would also "simply never announce mate". And there would be no need for special handling / interpretation of new command strings, and handshaking at startup to let the engine figure out if the GUI supports the new command, or that it should refrain from sending it. Engines supporting would still have to recognize their internal scores as mate scores, derive the dtm from it,and cast it into a special format before printing.