Polyglot keys for chess variants.

Discussion of chess software programming and technical issues.

Moderator: Ras

Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Polyglot keys for chess variants.

Post by Michel »

An important feature of the design of XBoard is that there is little need to remember such things
Well I prefer very much the command line interface to xboard. I think one of the strong points of xboard is that everything (well most things) can be set easily on the command line. The book interface looks too complicated to me.

I still don't see why -book "..." is not allowed to work (that's what icsdrone and cutechess-cli use). It would simply mean that whenever an engine (not a human) has to move and the position is in the book then xboard moves on its behalf.

I think this covers 99% of book uses. Power users can use the more refined options if they want to.
User avatar
hgm
Posts: 28483
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot keys for chess variants.

Post by hgm »

Well, this is historically grown, and partly a reflection that HasOwnBook seems to be a UCI standard option, that is supposed to suppress usage of the GUI book for that engine. This is all stuff I inherited from Winboard_x, and seems like UCI design leaking into XBoard.

The disadvantage of operating XBoard through the command-line is exactly the one you are stumbling on now: you have to know exactly how an option is called. (In this case: -polyglotBook FILENAME and -usePolyglotBook true|false.) Using the GUI interface allows you to simply browse through the options, and see what is offered. For the one-time effort of configuring new software, this is higly recommended.

That you need to set two options, a filename and a switch to enable its use, might seem inefficient. But I actually found it very useful, as it allows you to switch the book on and off without losing the filename, and having to retype it when you want to switch it back on. Remember that, even when you use the command line, the options are still persistent, so once you specified your favorite book, you can limit yourself to only specifying whether it should be used. (This is in fact so convenient that I now have added switching options for most persistently specified filenames in WinBoard, like the piece font or board texture bitmaps. Before, you could disable their use without losing the filename by prefixing the latter with *. Which was fine when you were editing the ini file, but required you to retype the full name when you wanted to get rid of the * prefix from the command line, which was very, very annoying. Especially after the settings file was relocated to a difficult-to-find place. )

What seems a duplication of controls is that there both is a global option to turn the book on or off, as well as per-engine options. But he global option is persistent, so setting -usePolyglotBook true once can make you pretend for the rest of your life that the option does not exist. As the volatile per-engine switches turn book use off by default, even an invalid book filename would never hurt you unless you explicitly try to use book. (Which then would be your own fault.) So it could indeed be argued that XBoard should install itself by default configured to use GUI book, even though no Polyglot book is included in the install package.
Last edited by hgm on Fri Sep 14, 2012 1:57 pm, edited 1 time in total.
Daniel Shawul
Posts: 4186
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Polyglot keys for chess variants.

Post by Daniel Shawul »

@Daniel: The Polyglot format is meant to be a simple format that is trivial to parse. Polyglot books are not really meant to be processed much further (although Polyglot implements a form of merging, and you can edit move scores in scid).
This is funny. So adding header bytes makes it more complex than xoring all hashkeys in the book with predefined magic keys? If you save positions for different variants in different files, the format is still the _same_ while allowing you all sorts of post processing.You don't even need to predefine the keys for varinats ,you can just save book for 'variant crazy' and match it later when winboard sends us 'variant name'. This also goes very well the 'alien edition' approach where variants are invented at run time. You can just generate a book for your new variant in polyglot format and use it right away. If you have to use the xoring method for some reason I don't get, then atleast save the magic key right in the book so that winboard and other tools do not have to be recompiled with the new magic
key for the new variant.

The most severe problem with your approach ,as I mentioned already, is you basically throughout all post procecessing by xoring with magic key as you never have anyway way of knowing to which variant the move belongs to. So the only way is to reconstruct books from scratch everytime. Xoring with predefined key for each variant causes problems for no apparent advantage.

IIRC scorpio has very similar format as polyglot's and I wouldn't want to throw away the book operations that it can do now if I extend it to Nebiyu. One would expect a better approach for a format to be shared among many people. For versioning and other format changes it is always good to reserve a few header bytes anyway. If you update the format with header bytes, the older books are still usable if we do version checking.

On a side note, I seem to recall that when I wanted to generate pawn bitbases with slices one file for each, I found it awkward that there are thousands of files when all can be merged in one. HG didn't have a problem with separate files IIRC...
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Polyglot keys for chess variants.

Post by Michel »

Using the GUI interface allows you to simply browse through the options,
Well it is exactly the same with a command line interface. You just read the manual page.
With the added advantage that you can search...
For the one-time effort of configuring new software, this is higly recommended.
It is only a one time effort if you use only one computer and never reinstall the os from scratch. I use many different computers (including virtual machines). So for each of these computers I would have to do the GUI configuration...and redo it after each clean reinstall.

Remembering xboard -book "..." (or perhaps invoking it from a script) would be easier to me.
User avatar
hgm
Posts: 28483
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot keys for chess variants.

Post by hgm »

The usage of a variant key by WinBoard was originally inspired because Polyglot books did not have a header (neither existing books, nor books that you would create with existing versions of Polyglot), as they were implied to be for normal Chess or a compatible pseudo-variant as Chess960. This, however, caused the problem that if I would set up Fairy-Max to play as an ICS book, naturally wanting it to use the GUI book for normal Chess, that it would create false hits on the book when someone challenged it in another variant employing same boards and pieces (like Berolina or Cylinder).

I needed a mechanism to prevent such false hits, and limiting book probing to normal / FRC at that point was no longer option, since I already had distributed a Xiangqi book with WinBoard as well, also not recognizable by a header. (There never is any mixup of Chess and Xiangqi keys, as they use different piece types for the ever-present Kings. So I realized there was a problem only later.)

So the variant key was added in a sense not to allow variant books to be merged with normal Chess books, but simply to prevent hits by spoiling the key it was trying to match. In most cases no opening theory exists for variants anyway, so I never seriously considered the possibility to have book moves for them. As such I never thought about the problem for unmerging a book. And it never occurred to me to make a complex mechanism for installing multiple books, and let WinBoard auto-switch between them, knowing that in practice not a single book would exists to switch to.

I don't think it is a very serious problem, though. Trying to cleanse a mixed Normal / Gothic book of its Gothic component just doesn't seem to be a thing you should want to do. If, for whatever reason, you would want to have such a multi-variant book, the proper procedure to create it would be to create books for the individual variants, and then merge those. You would not delete the originals after that. So if you wanted to drop a variant, you would simply redo the merging leaving it out.

It just seems an advantage that you can create mixed books. No one is forcing you to do it. So it can never be a disadvantage. Adding a single key in the hash is not a big deal.
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Polyglot keys for chess variants.

Post by Michel »

@Daniel: The variant keys are meant to make plurivariant books possible in a backwards compatible way. Nobody is preventing you from using single variant books if you prefer them.
User avatar
hgm
Posts: 28483
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot keys for chess variants.

Post by hgm »

Michel wrote:Well it is exactly the same with a command line interface. You just read the manual page.
With the added advantage that you can search...
The XBoard man pages always struck me as awful, but perhaps that is just a matter of taste. They seem organized as a reference manual, making it very hard to find anything when you don't already know what you are searching for. And by now it is very, very long.

Be honest to yourself: if it was that easy to find it in the manual, why are you asking me what the option is? It seems you gave up on using the GUI book rather than continuing your efforts to find it in the manual! While in fact searching for 'book' or 'Book' in the man file should have found you the the relevant options.

I think looking in the menu dialogs would on average be at least 5 times faster than looking for something in the manual you don't know already. The manual is organized exactly like the menus. Is it really so counter-intuitive to look for an option to set the book under Options -> Common Engine? (If the alternatives are mostly like Options -> Time Control, or Options -> ICS?)
Remembering xboard -book "..." (or perhaps invoking it from a script) would be easier to me.
Well, I am pretty sure that now you know that it is -polyglotBook instead, you will not ever forget it either. Note that there also is an option -discourageOwnBooks true|false, which can be used to invert the default of the HasOwnBook options in tournaments, so that all engines would automatically use book, even if they are not explicitly installed to do so. (I don't think this would work without specyfying a tourneyFile, though.)
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Polyglot keys for chess variants.

Post by Michel »

Be honest to yourself: if it was that easy to find it in the manual, why are you asking me what the option is?
Well if it had been an easy interface it would have been easy to find. I would have searched for "book" and found the appropriate option and remembered it.

I still think that a single short command line option should be all that is needed to enable a book (cutechess-cli manages to do it that way and icsdrone as well).

But I think everything has been said now about this subject. No need to discuss it further.
Michel
Posts: 2292
Joined: Mon Sep 29, 2008 1:50 am

Re: Polyglot keys for chess variants.

Post by Michel »

Here is my proposal for a header for polyglot books.

http://hardy.uhasselt.be/header.pdf

Following Daniel's suggestion I decided to include the variant names in the header since that is useful metadata.
User avatar
hgm
Posts: 28483
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot keys for chess variants.

Post by hgm »

Wouldn't it be better to separate major and minor version number by a period, rather than a linefeed (i.e. make it one field)? Then it would be immediately obvious for someone printing the file as text that he is dealing with version numbers.