Some xboard issues

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Some xboard issues

Post by hgm »

Evert wrote:A completely unrelated question/comment (I'd make a new post, but I'm writing this one anyway): is there any systematics in whether variants in XBoard use generic or variant-specific names for pieces? I originally had "ferz" in Makruk, but I changed it to "met", with the rationale that this is the actual name of the piece and it also happens to be the name that XBoard uses. For those exact same reasons, I changed to "Janus" rather than "Archbishop".
However, for Grand chess, the "proper" names for the pieces are "cardinal" and "marshal", but XBoard uses the generic names "Archbishop" and "Chancellor"...
Is it too late to apply some sort of standardisation? I think I would prefer consistently using variant-specific names rather than generic names (but otoh I do prefer "rook" to "chariot" or "boat"), but consistently using either scheme is preferable to things being different for every variant...
Or at the very least, is there some sort of documentation somewhere for the names of pieces across variants?
Or maybe the best solution is to not settle on either (given the already inconsistent set we have now and existing engines that use it) but allow an engine to say "look, this piece that moves like a knight+bishop and that you call an Archbishop, I call a Cardinal" and then XBoard would do some translation when talking to the engines. This could even be done implicitly if the engine sends the starting position using a setup command.
I know it's a relatively minor issue, but I find it annoying. ;)
My current policy is to use variant-specific names as much as possible, and the pieceToCharTable was introduced as a feature to make that possible (when my initial attempt to use global piece IDs ran aground). Because this makes it possible to write PGN that is more meaningful to people used to that variant, and enhances the chances of being able to read existing PGN (if they don't obfuscate the coordinates).

But when communicating with ICS, I have to follow ICS standards, as it would be a quite complex change to use different pieceToCharTable for ICS compared to PGN and engine. (And you would not be able to read PGN mailed to you by the ICS, etc.) For this reason Shatranj uses B for Elephant and Q for Ferz, because the ICC implemtation ofit has made that the "industry standard" for Shatranj.

The problem is compounded by the fact that ICS do not use pieceToCharTables that can be set per variant, and need the same piece (= code used on the internal board) to be indicated by the same letter in every variant. So to display a piece that moves like Archbishop as 'C' on the board, I would have to introduce a new piece type, add it to the move generator and the move checker, add it to the print routine for all 12 board styles, add code to disambiguate it based on the variant in the move parser, which has separate disambiguation code for moved pieces and promotion pieces. The problem is that the ICS's idea of a 'variant' is normal chess from a non-standard opening position, so there was never need to be aware of what variant you are playing anywhere except in the routine that sets up the opening position, and often the informationisnoteasily available at a place where you need it.

When implementing Grand Chess I did not feel like adding yet another piece that moved as an Archbishop just to be able to be able to indicate it by a C, and expanding the code for disambiguating 'C'. Also because the code at that time was written to assume K was the highest piece (used in array sizes), so that I could not increase the number of pieces without increasing the King code, which would basically invalidate all stored games. So to get it done quickly, I used the already existing Capablanca pieces and their ID.

I must say that I do have some doubt about the desirability to allow each variant to pick yet another name, or at least ID letter, for the same piece. E.g. a KQNKW end-game from Spartan Chess is entirely equivalent to a KQNKH end-game in Seirawan Chess, but the different naming would make it more difficult to look up positions in a database. Especially for 10x8 sub-variants it seems madness to name the B+N and R+N pieces in a zillion different ways. In Chess960 we also do not rename 'King' to 'President' when it happens to start on the d-file, 'Sultan' when it starts on the c-file,'Sjah' when it starts on the b-file...
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

hgm wrote: My current policy is to use variant-specific names as much as possible, and the pieceToCharTable was introduced as a feature to make that possible (when my initial attempt to use global piece IDs ran aground). Because this makes it possible to write PGN that is more meaningful to people used to that variant, and enhances the chances of being able to read existing PGN (if they don't obfuscate the coordinates).
Those are additional reasons.
But when communicating with ICS, I have to follow ICS standards, as it would be a quite complex change to use different pieceToCharTable for ICS compared to PGN and engine. (And you would not be able to read PGN mailed to you by the ICS, etc.) For this reason Shatranj uses B for Elephant and Q for Ferz, because the ICC implemtation ofit has made that the "industry standard" for Shatranj.
Aha, I was wondering about that.
The problem is compounded by the fact that ICS do not use pieceToCharTables that can be set per variant, and need the same piece (= code used on the internal board) to be indicated by the same letter in every variant. So to display a piece that moves like Archbishop as 'C' on the board, I would have to introduce a new piece type, add it to the move generator and the move checker, add it to the print routine for all 12 board styles, add code to disambiguate it based on the variant in the move parser, which has separate disambiguation code for moved pieces and promotion pieces. The problem is that the ICS's idea of a 'variant' is normal chess from a non-standard opening position, so there was never need to be aware of what variant you are playing anywhere except in the routine that sets up the opening position, and often the informationisnoteasily available at a place where you need it.
Right.
What about the idea of having a translation between XBoard and ICS for piece types?
When implementing Grand Chess I did not feel like adding yet another piece that moved as an Archbishop just to be able to be able to indicate it by a C, and expanding the code for disambiguating 'C'. Also because the code at that time was written to assume K was the highest piece (used in array sizes), so that I could not increase the number of pieces without increasing the King code, which would basically invalidate all stored games. So to get it done quickly, I used the already existing Capablanca pieces and their ID.
I understand the consideration; as I said, it's just a bit annoying.
I must say that I do have some doubt about the desirability to allow each variant to pick yet another name, or at least ID letter, for the same piece. E.g. a KQNKW end-game from Spartan Chess is entirely equivalent to a KQNKH end-game in Seirawan Chess, but the different naming would make it more difficult to look up positions in a database. Especially for 10x8 sub-variants it seems madness to name the B+N and R+N pieces in a zillion different ways.
Absolutely, which is a good argument to always use generic names, or at least have the engine recognise generic names in addition to specific names.

Part of the problem is of course that the protocol has grown into its current form rather than been designed ab-initio with variant support. Perhaps it's a good idea to think/discuss a bit more about ways in which the protocol could be extended for better support for variants, when it comes to piece names, board sizes, how to use holdings, starting positions... and formalise it. Perhaps protocol version 3 could add new facilities for variants?
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

Evert wrote:
I just pushed a debug version to the aliennew branch. Could you try to run that for the clock problem? It should print some more info on the routine that adds the time quota.
Will do!
Here is the relevant section of the log file:

Code: Select all

xboard master-20110618 + xsjaak-399-universal
Reset(1, 0) from gameMode 0
recognized 'normal' (-1) as variant normal
GameEnds(0, (null), 2)
shuffleOpenings = 0
TC string = ':40/10'
mps=40 tc=10000 inc=0
TC string = ':40/10'
mps=40 tc=10000 inc=0
291 >first : xboard
protover 2
shuffleOpenings = 0
[...]
3873 >second: xboard
protover 2
[...]
New game (0): Sjaak 399-Sjaak 399 (w)
3895 >first : computer
3895 >second: computer
TC string = ':40/10'
mps=40 tc=10000 inc=0
TC string = ':40/10'
mps=40 tc=10000 inc=0
time odds: 1.000000 1.000000 
3895 >first : time 1000
3896 >first : otim 1000
book hit = (NULL)
3896 >first : go
nps: w=-1, b=-1
[...]
4120 <first &#58; move g1f3 
machine move 0, castling = 7 0 4 7 0 4
trial 6,1,5,3  type 2121
7 0 4 7 0 4 Legality test? g1f3
7 0 4 7 0 4 Legality test? g1f3
&#40;7,0&#41; &#40;0,0&#41; &#40;4,0&#41; &#40;7,7&#41; &#40;0,7&#41; &#40;4,7&#41; castling rights
CoordsToAlgebraic, piece=1 &#40;6,0&#41;-&#40;5,2&#41; - PARTIAL-1=ffffffff
check B in&#58; last=&#40;0,0&#41; rem=&#40;9776,188978571024&#41;
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
check B out&#58; last=(-9776,188978571024&#41; rem=&#40;9776,188978571024&#41;
MateTest&#58; K=1, my=16, his=16
move&#58; g1f3
, parse&#58; Nf3 (
)
MateTest&#58; K=1, my=16, his=16
repeat test fmm=1 bmm=0 ep=-4, reps=6
1 ep=-4
0 ep=-4
time odds&#58; 1.000000 1.000000 
4120 >second&#58; time 18897857102
4120 >second&#58; otim 977
book hit = &#40;NULL&#41;
4120 >second&#58; g1f3
4120 >second&#58; go
It looks like the variable blackTimeRemaining isn't initialised, or gets an uninitialised value. I tried compiling with warnings on to see if there was anything obvious there, but there's so much output I got a bit lost (the code is not the easiest to follow or the best designed code I've ever had to work with... at least it's not written in FORTRAN 77 though). I also tried debugging with GDB, which again didn't help. I could try running through valgrind and see whether the problem shows up there, but that'll be very slow.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

Any news?
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Some xboard issues

Post by hgm »

I was on La Palma the past two weeks, without the laptop I use for development, and connectivity only through a smartphone. So not yet, but at least I can now start working on it again.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

hgm wrote:I was on La Palma the past two weeks, without the laptop I use for development, and connectivity only through a smartphone.
Sounds nice!
So not yet, but at least I can now start working on it again.
Ok.
Let me know if there's something I can run to help debug the issue again.
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Some xboard issues

Post by hgm »

It seems there is a type mismatch somewhere. 188978571024 = 0x2c00002710, and 0x2710 = 10000, what it should be. Probably TimeRemaining is declared as int in some files, and long int in others. In 32-bit compiles this is the same. But in 64-bit mode one is 32-bit, the other 64...

Not sure why it should ever be long int. Are there platforms where plain int is only 16 bits? XBoard is not supposed to work in 16-bit mode... TimeRemaining should be protected from becoming very short in 32-bit mode, but the only thing declaring it as long int seem to acheive is making it ridiculously long in 64-bit mode!
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

hgm wrote:It seems there is a type mismatch somewhere. 188978571024 = 0x2c00002710, and 0x2710 = 10000, what it should be. Probably TimeRemaining is declared as int in some files, and long int in others. In 32-bit compiles this is the same. But in 64-bit mode one is 32-bit, the other 64...
Ah, that could explain something: my system is 64 bit.
Shouldn't there just be a single declaration in a header file somewhere?
Not sure why it should ever be long int. Are there platforms where plain int is only 16 bits?
Not that I know of, but in theory there could be (that said, I suspect it probably was 16 bit on DOS and early versions of Windows).
XBoard is not supposed to work in 16-bit mode... TimeRemaining should be protected from becoming very short in 32-bit mode, but the only thing declaring it as long int seem to acheive is making it ridiculously long in 64-bit mode!
Certainly in 32 bit mode an int should not be 16 bits. I think it shold be safe to use "int" throughout (or better yet, put the declaration in a headerfile so there never is a mis-match).
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Some xboard issues

Post by hgm »

Evert wrote:Shouldn't there just be a single declaration in a header file somewhere?
Ideally it should, but XBoard is pretty sloppy in this respect. Strange thing here is that the original assignement printing and usage all take place in the same file backend.c, which also declares it as 'long'. I can't see anything wrong with the code in this respect.

Of course an alternative explanation would be that there is an out-of-range pointer somewhere used to store data, which happens to point to the high word of blackTimeRemaining in your compile. It coul also be an overflow from a neigboring variable that is declared too narrow, or a neighboring array that overwrites its bounds. The value 0x2c=44 of the high word corresponds to the board code for an EmptySquare.

I made a new commit to aliennew, which prints blackTimeRemaining with every engine input or output, so we should be able to see when approximately it changes. Could you run that?
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Some xboard issues

Post by Evert »

Here it is:

Code: Select all

xboard master-20110618 + xsjaak
Reset&#40;1, 0&#41; from gameMode 0
recognized 'normal' (-1&#41; as variant normal
GameEnds&#40;0, &#40;null&#41;, 2&#41;
shuffleOpenings = 0
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
ResetClocks&#58; btr=10000 100243208 100465678 1004617c0
713 >first &#58; &#40;10000&#41; xboard
protover 2
shuffleOpenings = 0
811 <first &#58; &#40;10000&#41; #&#91;-&#93;>
811 <first &#58; &#40;10000&#41; feature setboard=1 time=1 sigint=0 colors=0 highlight=1 analyze=1 ping=1 memory=1 pause=1 nps=1 myname="Sjaak 427&#58;442M" myversion="&#91;427&#58;442M Dec 21 2011&#93; &#40;x86_64&#41;"
811 >first &#58; &#40;10000&#41; accepted setboard
811 >first &#58; &#40;10000&#41; accepted time
811 >first &#58; &#40;10000&#41; accepted sigint
811 >first &#58; &#40;10000&#41; accepted colors
811 >first &#58; &#40;10000&#41; accepted highlight
811 >first &#58; &#40;10000&#41; accepted analyze
811 >first &#58; &#40;10000&#41; accepted ping
811 >first &#58; &#40;10000&#41; accepted memory
811 >first &#58; &#40;10000&#41; accepted pause
811 >first &#58; &#40;10000&#41; accepted nps
811 >first &#58; &#40;10000&#41; accepted myname
812 >first &#58; &#40;10000&#41; rejected myversion
812 <first &#58; &#40;10000&#41; feature variants="normal,berolina,spartan,amazon,knightmate,maharaja,makruk,shatranj,fischerandom,seirawan,capablanca,gothic,10x8+0_janus,caparandom,courier,xiangqi,grand,10x8+0_fairy,10x10+0_fairy,9x9+0_fairy,8x8+2_fairy,fairy"
812 >first &#58; &#40;10000&#41; accepted variants
812 <first &#58; &#40;10000&#41; feature option="Variant configuration file -file ../variants.txt"
812 >first &#58; &#40;10000&#41; accepted option
812 <first &#58; &#40;10000&#41; feature option="Variant fairy selects -combo Normal &#40;8x8&#41; /// Amazon &#40;8x8&#41; /// Grand &#40;10x10&#41; /// Indian Grand &#40;10x10&#41; /// Maharaja &#40;8x8&#41; /// Pocket Knight &#40;8x8+1&#41; /// FIDE Chess &#40;8x8&#41; /// Los Alamos Chess &#40;6x6 on 8x8&#41; /// Legan's Chess &#40;8x8&#41; /// Chancellor Chess &#40;9x9&#41; /// Chancellor Chess &#40;8x8&#41; /// Ninth Century Indian Chess &#40;8x8&#41; /// Halma &#40;8x8&#41; /// Fox and Geese &#40;8x8&#41; "
812 >first &#58; &#40;10000&#41; accepted option
812 <first &#58; &#40;10000&#41; feature option="Variant seirawan selects -combo None /// Pocket Knight /// Burmese"
812 >first &#58; &#40;10000&#41; accepted option
812 <first &#58; &#40;10000&#41; feature option="Load variants -reset"
812 >first &#58; &#40;10000&#41; accepted option
812 >first &#58; &#40;10000&#41; rejected option Load variants -reset
812 <first &#58; &#40;10000&#41; feature done=1
812 >first &#58; &#40;10000&#41; accepted done
813 >first &#58; &#40;10000&#41; memory 68
813 >first &#58; &#40;10000&#41; new
random
813 >first &#58; &#40;10000&#41; level 40 0&#58;10 0
813 >first &#58; &#40;10000&#41; post
813 >first &#58; &#40;10000&#41; hard
813 >first &#58; &#40;10000&#41; easy
813 >first &#58; &#40;10000&#41; ping 1
820 <first &#58; &#40;10000&#41; pong 1
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
ResetClocks&#58; btr=10000 100243208 100465678 1004617c0
time odds&#58; 1.000000 1.000000 
6535 >first &#58; &#40;10000&#41; time 1000
6535 >first &#58; &#40;10000&#41; otim 1000
book hit = &#40;NULL&#41;
6535 >first &#58; &#40;10000&#41; go
nps&#58; w=-1, b=-1
6538 <first &#58; &#40;10000&#41;   2    5     0        68 Nb1-c3 
6546 <first &#58; &#40;10000&#41;   3    9     1       574 Ng1-f3 Ng8-h6 
6548 <first &#58; &#40;10000&#41;   4   13     1       691 Ng1-f3  e7-e6 Nb1-c3 
6565 <first &#58; &#40;10000&#41;   5   17     2      2381  e2-e4  e7-e6 Qd1-f3 Ng8-h6 
6626 <first &#58; &#40;10000&#41;   6    5     9      9278  e2-e3 Ng8-f6 Nb1-c3  c7-c6 Ng1-f3 
6713 <first &#58; &#40;10000&#41;   7    8    17     20661 Nb1-c3 Nb8-c6  d2-d4  d7-d5 Ng1-f3 Ng8-h6 
6714 <first &#58; &#40;10000&#41; move b1c3 
machine move 0, castling = 7 0 4 7 0 4
trial 1,1,2,3  type 2121
7 0 4 7 0 4 Legality test? b1c3
CoordsToAlgebraic, piece=1 &#40;1,0&#41;-&#40;2,2&#41; - PARTIAL-1=ffffffff
check B in&#58; last=&#40;0,0&#41; rem=&#40;9821,188978571024&#41;
TC string = '&#58;40/10'
mps=40 tc=10000 inc=0
check B out&#58; last=(-9821,188978571024&#41; rem=&#40;9821,188978571024&#41;
MateTest&#58; K=1, my=16, his=16
move&#58; b1c3
, parse&#58; Nc3 (
)
MateTest&#58; K=1, my=16, his=16
repeat test fmm=1 bmm=0 ep=-4, reps=6
1 ep=-4
0 ep=-4
AnimateMove&#58; piece 1 hops from 1,0 to 2,2 
time odds&#58; 1.000000 1.000000 
7326 >first &#58; &#40;188978570929&#41; time 18897857092