Could anyone make me a Windows compile...?

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Could anyone make me a Windows compile...?

Post by Evert »

mar wrote:Ok the fix is rather simple. A typo I guess:
keyboard_input_waiting:

Code: Select all

   if (virgin) {
      virgin = false;    // instead of virgin = true
now I get the expected 13%. however when run manually from the console it's still using only a fraction of CPU time.
EDIT: +100 elo for Sjaak on Windows now in blitz I guess :)
Wow, well done!

You've just fixed a bug in every chess program I've ever written. I have some other bugfixes locally, so it may make sense to push out RC2...
mar
Posts: 2681
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Could anyone make me a Windows compile...?

Post by mar »

Evert wrote:I have some other bugfixes locally, so it may make sense to push out RC2...
Sure, I guess after you agree with Ferdy on perft :) You can simply PM me when ready and I will recompile.
mar
Posts: 2681
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Could anyone make me a Windows compile...?

Post by mar »

I've just tried Jazz and it utilizes full cpu so perhaps the bug only affects Sjaak in a negative way.
User avatar
hgm
Posts: 28480
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Could anyone make me a Windows compile...?

Post by hgm »

Sorry to keep pestering you:

Someone wanted to have an XBoard engine for Goro Goro Shogi, and I recommended him Sjaak II, to configure it himself. Out of curiosity I peeked in the variants.txt file, to see how difficult this would be. Well, it seemed reasonably straightforward, except...

You do mention holdings, demotion on capture and dropping there, but it does not specify how this should be enabled. (E.g. should it be specified for each piece separately, or is it a game-global flag, and what are the required keywords and syntax.) I also could not find any drop game amongst the examples either.

Could you fix that in the description?

A second point that was not entirely clear to me:
Is castling reserved for royal pieces only, or can any piece set the castle flag? I ask because I recently discovered that in Omega Chess there is this silly feature where you can castle a Queen with a Rook. (They call it 'guarding'.) Of course the problem ther is that both the Queen and the Rook move would always be legal as well, so you have to enter it as QxR. Can do Sjaak Omega Chess anyway, or is 12x12 too big?

I noticed that the examples do not set a castling flag on the Rooks. How is the castling partner determined? Is it always a Rook, or is it always the corner piece?
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Could anyone make me a Windows compile...?

Post by Evert »

hgm wrote:Sorry to keep pestering you:
No bother, keep'm coming!
Someone wanted to have an XBoard engine for Goro Goro Shogi, and I recommended him Sjaak II, to configure it himself. Out of curiosity I peeked in the variants.txt file, to see how difficult this would be. Well, it seemed reasonably straightforward, except...

You do mention holdings, demotion on capture and dropping there, but it does not specify how this should be enabled. (E.g. should it be specified for each piece separately, or is it a game-global flag, and what are the required keywords and syntax.) I also could not find any drop game amongst the examples either.
Oops, it seems I didn't include setting the rules for dropping or holdings in the parser for the config file. I'll fix that for RC2.
It's a per-game option, by the way.
A second point that was not entirely clear to me:
Is castling reserved for royal pieces only, or can any piece set the castle flag?
Any piece should be able to get the castle flag. There should be no assumption in the code that the main castle piece should be royal (but I haven't tried it).
However, only one piece can be given the castle flag. The way the castling move is defined is by specifying the from-square of the castle piece and the destination square, along with the square holding the partner piece ("Castle white e1-g1 with h1" is short castling for white). The partner piece is then identified from the starting position.
I ask because I recently discovered that in Omega Chess there is this silly feature where you can castle a Queen with a Rook. (They call it 'guarding'.) Of course the problem ther is that both the Queen and the Rook move would always be legal as well, so you have to enter it as QxR. Can do Sjaak Omega Chess anyway, or is 12x12 too big?
12x12 is too big. There are a number of arrays that have their size hard-coded to 128 squares maximum, but more importantly the move encoding also assumes this. I'm not sure there is space to extend that to 256 squares…

There is a work-around though: the board is 10x10 with 4 extra squares in the corners. Hacking the board masks and leaper movement tables is relatively easy. That still leaves sliders, but I may have an idea on how to do that too. Either way, hacking the movement tables directly cannot be done from the config file.
I noticed that the examples do not set a castling flag on the Rooks. How is the castling partner determined? Is it always a Rook, or is it always the corner piece?
The castle flag is only set on the piece that initiates castling (ie, the king). The castling partner is then identified by the square it's on in the line that describes the castling move, that this is missing in the examples is probably an oversight. It might make sense to define a default of "king moves two steps and castles with corer piece".
User avatar
hgm
Posts: 28480
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Could anyone make me a Windows compile...?

Post by hgm »

It is not that I am particularly fond of Omega Chess, but I don't quite see the need fo having only a single piece that can castle. What would go wrong if a game definition would contain both the lines:

Castle: e1-g1 with h1
Castle: d1-f1 with h1

and set the castle flag on both K and Q? The from-square defines which rule is for which piece. Why would a castle flag need to be explicitly specified with the piece anyway? The "Castle:' line mentions the from square, and it should be known which piece is on it. (I must admit that none of the examples in the variants.txt file conatins a Castle: line, though. Is a default assumed in that case? How does the default depend on the board width?)

If two pieces have castling ability, it could just generate the castlings that have the correct from-square. It has to check plenty conditions before the castling can be allowed, and the from square could be made one of them.

BTW, if a non-royal piece is given castling rights, would it be allowed to castle over attacked squares?
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Could anyone make me a Windows compile...?

Post by Evert »

hgm wrote:It is not that I am particularly fond of Omega Chess, but I don't quite see the need fo having only a single piece that can castle. What would go wrong if a game definition would contain both the lines:

Castle: e1-g1 with h1
Castle: d1-f1 with h1
The move generator uses three bitmasks for the generation of castling moves: one that indicates which pieces are required to be virgin, one that indicates which squares should be empty for castling to be possible and one that indicates which squares should be safe (not attacked) in order for castling to be legal. There are two sets of masks for white and two for black (so four sets in total). These masks are (re)calculated based on the FEN to accommodate Chess960-style castling and they keep the move generator fairly agnostic about what is actually going on: it just knows to move some pieces from where they are to a new location, and it pulls all relevant data from tables.

The reason this doesn't work with more than one piece is that the castle masks are defined for each game, rather than for each piece. That wouldn't be hard to change though. The origin for this is just that I used that design in Jazz and reused it for Sjaak. It never really occurred to me to make it a per-piece option (which I did actually do for promotion zones).
Why would a castle flag need to be explicitly specified with the piece anyway?
The reason the flag exists is so the move generator can test whether to even try to generate castling moves for a particular piece before it starts testing bitmasks. You're right that it's implied by the definition of the castling move, so setting it explicitly should be redundant.
(I must admit that none of the examples in the variants.txt file conatins a Castle: line, though. Is a default assumed in that case? How does the default depend on the board width?)
There currently is no default, which means that the missing Castle: line is really a bug. I have to check if this is something that was already present in the older version of Sjaak, or whether this is something I introduced recently. For the default I would specify that for short castling the king moves next to the corner, long castling moves the same number of squares in the opposite direction and castling is done with the corner piece.

But as I said, it doesn't actually do this now.
BTW, if a non-royal piece is given castling rights, would it be allowed to castle over attacked squares?
No, but that could easily be changed by setting the "must be safe" masks to 0 if the castling piece is non-royal.

So really the reasons this doesn't work properly right now are
  • Castle masks are defined per-game rather than per-piece (changing this is straightforward)
  • Castling with non-royal pieces would be disallowed if it passes over "unsafe" squares (very easy to fix)
  • By default, all castling moves would be output as O-O or O-O-O, independent of whether the moving piece is a king or (say) a queen.
The last point is probably the most annoying one in practice. The reason SjaakII prefers O-O and O-O-O for castling moves was originally that this allows it to treat Chess960 and ortho-chess as the same game: XBoard protocol defines O-O and O-O-O as castling notation for Chess960 and understands it for orthochess. In practice this isn't so straightforward because UCI interfaces doesn't understand O-O/O-O-O and (worse) don't understand KxR if the variant is not set to Chess960 (I ran into this awhile back).
Even if I get rid of O-O/O-O-O though it'll be annoying to work out what exactly to send to the GUI for a (non-standard) castling move since the easy alternative (always sending "KxR", ugly as it is) doesn't always work...
User avatar
hgm
Posts: 28480
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Could anyone make me a Windows compile...?

Post by hgm »

Evert wrote:The move generator uses three bitmasks for the generation of castling moves: one that indicates which pieces are required to be virgin, one that indicates which squares should be empty for castling to be possible and one that indicates which squares should be safe (not attacked) in order for castling to be legal. There are two sets of masks for white and two for black (so four sets in total). These masks are (re)calculated based on the FEN to accommodate Chess960-style castling and they keep the move generator fairly agnostic about what is actually going on: it just knows to move some pieces from where they are to a new location, and it pulls all relevant data from tables.
Well, from this it sounds like the limitation is just that you only allow two sets of masks per side. It seems useful to increase that anyway, because some variants have castling rules where you can choose how many steps the King takes in the direction of the Rook (e.g. whether to Q-side castle by e1-b1 or e1-c1). I see no reason why some masks could not indicate virginity of the King (+ applicable Rook) is required, and other masks would require virginity of the Queen (+ Rook). They already differ in the virginity requirement for the Rooks.

I guess it depends a little on how you treat castling generation. Many engines do not tie that to generation of moves with a certain piece, but just have a separate stage 'generate all castlings' (skipped in check-evasion nodes). In such a design it would not really matter which pieces can castle, or not. You would just loop through all mask sets for the stm, testing whatever they specify you should test, and if it passes the test, the table should tell you which pieces to move and where.
So really the reasons this doesn't work properly right now are
  • Castle masks are defined per-game rather than per-piece (changing this is straightforward)
  • Castling with non-royal pieces would be disallowed if it passes over "unsafe" squares (very easy to fix)
  • By default, all castling moves would be output as O-O or O-O-O, independent of whether the moving piece is a king or (say) a queen.
The last point is probably the most annoying one in practice. The reason SjaakII prefers O-O and O-O-O for castling moves was originally that this allows it to treat Chess960 and ortho-chess as the same game: XBoard protocol defines O-O and O-O-O as castling notation for Chess960 and understands it for orthochess. In practice this isn't so straightforward because UCI interfaces doesn't understand O-O/O-O-O and (worse) don't understand KxR if the variant is not set to Chess960 (I ran into this awhile back).
Even if I get rid of O-O/O-O-O though it'll be annoying to work out what exactly to send to the GUI for a (non-standard) castling move since the easy alternative (always sending "KxR", ugly as it is) doesn't always work...
Ah yes, this is very annoying. Using O-O outside Chess960 does work in XBoard, but actually is a protocol violation.

Perhaps the protocol notation for each castling should be made configurable through the Castling: line? For input moves you could then always compare them against each entry in the castling table, and if it matches one, and the corresponding castling is a legal move, it must mean that castling, (as otherwise the notation would have been defined in an ambiguous way), and you can encode it by whatever internal format you are using (which would also be in the table). Only if it doesn't match, or the one it matches isn't legal, you would subject the input move to normal parsing. On output you would just recognize the internal castling format, and use the tabulated text string for it.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Could anyone make me a Windows compile...?

Post by Evert »

hgm wrote: Well, from this it sounds like the limitation is just that you only allow two sets of masks per side. It seems useful to increase that anyway, because some variants have castling rules where you can choose how many steps the King takes in the direction of the Rook (e.g. whether to Q-side castle by e1-b1 or e1-c1). I see no reason why some masks could not indicate virginity of the King (+ applicable Rook) is required, and other masks would require virginity of the Queen (+ Rook). They already differ in the virginity requirement for the Rooks.

I guess it depends a little on how you treat castling generation. Many engines do not tie that to generation of moves with a certain piece, but just have a separate stage 'generate all castlings' (skipped in check-evasion nodes). In such a design it would not really matter which pieces can castle, or not. You would just loop through all mask sets for the stm, testing whatever they specify you should test, and if it passes the test, the table should tell you which pieces to move and where.
I guess that could be a good idea. I'll look into that.
I try to limit as much as possible testing for things I don't need, but in this case it would take a test out of the inner-loop (meaning it's done once per piece-type) and move it outside the loop, meaning it's done once per castling move per move gen, and then possibly only if we're not generating captures-only, so that would be better.
Ah yes, this is very annoying. Using O-O outside Chess960 does work in XBoard, but actually is a protocol violation.
Huh?
Even though all versions of this protocol specification have indicated that xboard accepts SAN moves, some non-xboard interfaces are known to accept only coordinate notation. See the Idioms section for more information on the known limitations of some non-xboard interfaces. It should be safe to send SAN moves if you receive a "protover 2" (or later) command from the interface, but otherwise it is best to stick to coordinate notation for maximum compatibility. An even more conservative approach would be for your engine to send SAN to the interface only if you have set feature san=1 (which causes the interface to send SAN to you) and have received "accepted san" in reply.
From http://www.gnu.org/software/xboard/engine-intf.html
Sure, it says it's a dangerous thing to do because not all interfaces support it, but it also explicitly says that the protocol spec says that SAN moves are allowed, and that they should be safe to use in protover 2 (why that would be is a complete mystery to me, and the last part makes no sense at all since the GUI sending you SAN moves doesn't imply that it would understand them if you sent them back; it would be deliberately malicious if it didn't, but maybe people who set SAN=1 deserve that sort of punishment).

Ok, so I would never have designed the protocol this way myself, but it's a part that I found it very convenient to abuse.
Perhaps the protocol notation for each castling should be made configurable through the Castling: line? For input moves you could then always compare them against each entry in the castling table, and if it matches one, and the corresponding castling is a legal move, it must mean that castling, (as otherwise the notation would have been defined in an ambiguous way), and you can encode it by whatever internal format you are using (which would also be in the table). Only if it doesn't match, or the one it matches isn't legal, you would subject the input move to normal parsing. On output you would just recognize the internal castling format, and use the tabulated text string for it.
Well, the move parsing is on the list of things to completely rewrite (it's a mess). I'll think about how I handle this.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Could anyone make me a Windows compile...?

Post by Evert »

hgm wrote: Someone wanted to have an XBoard engine for Goro Goro Shogi, and I recommended him Sjaak II, to configure it himself. Out of curiosity I peeked in the variants.txt file, to see how difficult this would be. Well, it seemed reasonably straightforward, except...

You do mention holdings, demotion on capture and dropping there, but it does not specify how this should be enabled. (E.g. should it be specified for each piece separately, or is it a game-global flag, and what are the required keywords and syntax.) I also could not find any drop game amongst the examples either.

Could you fix that in the description?
This is now fixed in the 1.0RC2 release, which will also correctly parse those options from the configuration file. I've added Judkins Shogi as an example too.

I don't have Windows binaries up yet though.
I noticed that the examples do not set a castling flag on the Rooks. How is the castling partner determined? Is it always a Rook, or is it always the corner piece?
This is now also fixed, and you can define a castling move for any piece (well, you already could), but you can still only define one "king-side" and one "queen-side" castling move for each side, which will be written "O-O" and "O-O-O" by default or by "<from><to>" or "KxR"/"RxK" in UCI mode. However, the code is mostly ready to extend this to any arbitrary number of castling moves for any piece, I just need to expand the arrays and handle the notation logic. What to do for notation is not so obvious though.
As you pointed out already, when "guarding" a Queen writing "d1c1" would be ambiguous, so it would have to be "QxR" but then the GUI does need to know how to handle this. If the queen moves differently (as a Ferz, say) so that "d1c1" is not ambiguous this wouldn't be needed, but I'd prefer for the notation to be at least somewhat agnostic with respect to how the pieces move.
I will probably just go for "QxR" for now; a multi-leg move including drops and pickups would be an alternative, but also ugly.