Kibitz score reporting in server play

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.

How should position evaluation scores be reported when kibitzing in server play?

Poll ended at Fri Mar 30, 2007 3:38 pm

In centipawns, from the viewpoint of White
3
13%
In centipawns, from the viewpoint of the program
7
29%
In decimal pawns, from the viewpoint of White
7
29%
In decimal pawns, from the viewpoint of the program
6
25%
Other
1
4%
 
Total votes: 24

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Kibitz score reporting in server play

Post by sje » Fri Mar 16, 2007 3:38 pm

How should position evaluation scores be reported when kibitzing in server play?

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Kibitz score reporting in server play

Post by sje » Sat Mar 17, 2007 7:34 am

How about this idea:

Say that a queen is worth one pound sterling. Then a rook would go for about eleven shilling and tuppence. Positional terms might be expressed in milli-farthings or micro-groats.

User avatar
Jaap Weidemann
Posts: 62
Joined: Mon Aug 14, 2006 1:47 am
Location: Stellenbosch, South Africa
Contact:

Re: Kibitz score reporting in server play

Post by Jaap Weidemann » Sat Mar 17, 2007 10:44 am

Sounds good to me!

:)

User avatar
Bill Rogers
Posts: 3562
Joined: Thu Mar 09, 2006 2:54 am
Location: San Jose, California

Re: Kibitz score reporting in server play

Post by Bill Rogers » Sat Mar 17, 2007 4:50 pm

Another good question. How do you determine whether the machines eval is awarding points for positional play instead of something else? As an example if a eval gives points for protecting another man on the board how would you know whether this was considered positional or a defensive measure?
I think some of your ideas sound good but just how to implement them is another question.
Bill

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Kibitz score reporting in server play

Post by sje » Sat Mar 17, 2007 5:14 pm

By "positional evaluation score" I mean the total of material, positional, and any other factors into one number.

--------

In the early computer chess literature, the score was almost always given from the White point of view and the custom remained for some time. The papers back then would label a search tree with alternating "min" and "max" ply. However, after the advent of the negamax formalism, it became much more common to see scores from the active color point of view. And that meant the program point of view at the search tree root.

As for score resolution: The authors of the Northwestern program Chess 4.x had it round all internal scores to the nearest quarter pawn. The claim was that this increased the number of A/B cutoffs. Maybe it did, but I wonder if anyone is doing anything similar today.

Symbolic's scores are thirty two bit signed integers and the base unit is one micropawn. Other than some special scores (like mate/lose), the micropawn number gets a floating divide by one million and is then output as a decimal number with up to three digits after the radix point. The gain from using floating output is that I can change the internal base unit without having it affect the external data presentation.

Odd Gunnar Malin
Posts: 179
Joined: Wed Mar 08, 2006 8:59 pm
Location: Norway, Vads?
Contact:

Re: Kibitz score reporting in server play

Post by Odd Gunnar Malin » Tue Mar 20, 2007 10:52 am

If you kibizes for the viewers, it have to be in the format they are used too. Eg. in decimalpoints from White's side of view, maybe even with an added +/= score. Ex. += (+0.45) 22...exf4 23.Rxf4

Internaly it should of course do what the protocol require (from engines point in both Winboard and UCI).

It is just as important to kibitzing the pv in fide notation and not in coordinate or long notation. The movenumber helps too to give an easy to understand pv-line.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Kibitz score reporting in server play

Post by sje » Tue Mar 20, 2007 1:27 pm

As mentioned before, Symbolic outputs some special scores in a non numeric format. A mate in four is "MateIn4" and a lose in twelve is "LoseIn12". There aren't any spaces because of the way Symbolic uses a single token for score I/O. The "MateInN" format is taken in part from Crafty's similar "MatN" output. But the program does not permit negation of either mating or losing scores, so there's no "-MatN" output. Internally, scores are not negated directly, but rather shifted up or down in the tree. So a "MateIn5" shifted up one ply is "LoseIn5"; the same score shifted down one ply is "LoseIn4".

Symbolic output a draw score as "Even".

When Symbolic locates a forced mate from its point of view, it kibitzes a separate line announcing this with a spelled number. E.g., "I have a checkmate in twenty seven moves." Admittedly, this could be an annoyance to a human opponent, but the I think most don't mind this too much. For a forced loss, there's no extra kibitz line.

--------

Other kibitz output:

1. The labeled PV in SAN with move numbers.

2. The labeled node frequency in Hz (usually KHz).

3. The search termination indicator form the following list (elide "CTSerachTerm"):

Code: Select all

  CTSearchTermActive,         // Still running
  CTSearchTermAllBadButOne,   // All moves except one are certain losing
  CTSearchTermAllCertain,     // All moves have certain scores
  CTSearchTermBadParams,      // Invalid input parameters
  CTSearchTermBookMove,       // Book move
  CTSearchTermCancelled,      // Cancellation
  CTSearchTermContingency,    // Contingency (insufficient time)
  CTSearchTermDepthLimit,     // Fixed depth limit
  CTSearchTermEarlyLose,      // Early lose detected in search
  CTSearchTermEarlyMate,      // Early mate detected in search
  CTSearchTermEasyMove,       // Easy move
  CTSearchTermFastMate,       // Sufficiently fast checkmate
  CTSearchTermHintMove,       // Quick move from hint facility
  CTSearchTermIterLimit,      // Fixed iteration count limit
  CTSearchTermIterSysMax,     // System maximum iteration
  CTSearchTermNoMoves,        // No moves at ply zero
  CTSearchTermNodeLimit,      // Fixed node count limit
  CTSearchTermNonSpecific,    // Non specific or unknown reason
  CTSearchTermOnlyMove,       // Only move
  CTSearchTermQuickMateIn1,   // Quick pick mate in one
  CTSearchTermQuickMateIn2,   // Quick pick mate in two
  CTSearchTermQuickMateIn3,   // Quick pick mate in three
  CTSearchTermRandomMove,     // Randomly selected move
  CTSearchTermSnapReply,      // SnapReply (oppo search nominal time)
  CTSearchTermTablebaseWin,   // Tablebase move (winning)
  CTSearchTermTimeLimitCand,  // Time limit (fixed/nominal time: candidate)
  CTSearchTermTimeLimitIter,  // Time limit (fixed/nominal time: iteration)
  CTSearchTermTimeLimitNode,  // Time limit (fixed/nominal time: per node)
  CTSearchTermUserRequest     // User requested


Vinvin
Posts: 4245
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Kibitz score reporting in server play

Post by Vinvin » Tue Mar 20, 2007 2:49 pm

I prefere a score in pawn unit, example : a rook = 5.00 ;
and from white point of view, it's more easy because you can understand score without looking at piece colour and programs give about the same score.

Zlaire

Re: Kibitz score reporting in server play

Post by Zlaire » Tue Mar 20, 2007 3:48 pm

Only a programmer would want the score in centipawns from the engines point of view. :)

Decimal from white's point of view is quite standard. Decimal because it's easier to quickly understand what the score is, and from white's point of view so you don't have to consider what side the engine is playing.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Kibitz score reporting in server play

Post by sje » Thu Mar 29, 2007 12:38 am

It looks like a 2 to 1 ratio in favor of scores reported from the program's point of view vs White's point of view. That's probably the reverse of what the opinion might have been a quarter century ago.

Idea 1: express the score as a ratio of strength; this is commonly used in military calculations.

Idea 2: express the score as an estimate of the number of moves remaining until win or loss.

Idea 3: express the score as a win/lose/draw triplet of probabilities.

Post Reply