The Carreblanca Rating List: Testers Wanted!

[hgm]

If you have an interest in 10x8 Chess, read on!

Now that the number of 10x8-capable WinBoard engines abounds, I would like to set up a rating list for them. To express that this rating will not focus on one particular variant of the many that are played on 10x8 with the pieces Chancellor and Archbishop, but which differ only in opening position (Capablanca, Gothic, Carrera, Bird, Embassy), I want to call this list by the name 'Carreblanca', being a contraction of the names of the original inventor of this type of Chess, Pietro Carrera, and the one who contributed most to its spreading (Jose Capablanca).

To determine the ratings we need many games between the engines. I have computer time to play a lot of these games myself, but of course no self-respecting rating list can allow programmers to test their own engines. And I have two engines, (like some other programmers have as well), so that eliminates a lot of games for me.

So I would like to make an inventory of who would be willing to participate in this project.

At the moment there are only eight 10x8 engines, but I know of several people that they are working on an engine as well. So to get an idea of the work involved, we should count on about 10 engines. To give each engine a decent number of games, it means they would have to play each other at least 10 times initially. (Newly arriving engines, and testing of improved versions will drive up the number of games to the desired 200 per engine later.) With 10 engines, that means 45x10 = 450 games in total.

I wanted to use the same testing conditions as CCRL, i.e. 40 moves per 40 'standard minutes', the actual time corrected for the hardware used. The rating aims at determining the strength of the search engine, rather than opening knowledge (which would be too variant-specific anyway). In addition, we could have a blitz list for 40/5'. A game typically lasts 60 moves, so that would make the game duration 2 hours (and 15 min for blitz) on standard hardware. (For comparison: a 2.4GHz Core 2 Duo would use only 80% of the standard time, and could play two games simultaneously, as we will be playing with ponder off.)

As this form of Chess has not been played often enough to have an enormous database of opening theory, I thought to solve the problem of multiple games between the same, possibly deterministic engines by starting from many different opening positions. The Carrera, Capablanca, Gothic, Bird and (mirrored) Embassy position would be obvious choices, and each of them could be played with white and black. This already gives 10 games. If more games are desirable, they could be randomly selected from (a subset of) the list of Capablanca Random Chess openings. To make sure all engines can play them, we should limit ourselves to positions with Rooks in the corners, and the Kings on the f-file. For an equal distribution of strength over the wings, I further wanted to impose the requirement that on each side of the King there is one Knight, and one Bishop (the Bishops being on squares of unlike color, of course). There are still 216 positions that satisfy these conditions. They could be played with black and white, for 432 different games, which seems more than enough. As it is not known if any of the positions happens to be strongly biased, I want to always play these games in pairs with reversed colors.

Perhaps we should start by testing the procedure, through making a 40/5' rating list. If unforseen problems turn up, we won't have wasted that much computer time. In this case we can easily afford more games, allowing us to try the shuffled openings.

[Kirill Kryukov]

Hi H.G., Good luck with assembling the team and testing! I will follow with interest!

[Spock]

I would be very interested in participating in this list. I was actually about to ask some questions myself with a view to maybe playing soem games, having seen all the activity about it lately.

I'm very keen on these chess variants, and of course you'll be aware that the CCRL FRC ratings list is 100% my games (although Kirill of course is the creator of the brilliant web site which makes it all possible, and moral support has been provided by other members !)

It will be a bit of a leap for me as I have never used Winboard, but I'm willing to learn if you have the time to give some guidance.

[hgm]

Well, that truly exceeds my highest hopes, to get a reaction from a person with your experience! Of course I am willing to provide every guidance you might need.

First of all, what do you think of my plan, of playing from differently shuffled opening positions, rather than using an external book as CCRL does? I am open to any suggestions.

As for using a WinBoard-type GUI: The way I use it is in combination with a tournament manager, PSWBTM, by Pradu Kannan. This tournament manager calls WinBoard with the necessary arguments to play the actual games, but is responsible for maintaining the database of engines and their preferred settings, and for conducting the tournaments (pairings, collecting the PGN files, keeping the score and ranking). It can handle gauntlets and round-robins, with an adjustable number of games per pairing.

Nice feature is that you can select to play the games from non-standard initial positions, given as either a PGN or a FEN file, and that you can select to play each position once or twice (with reversed colors). Pradu has made a special version of PSWBTM for me that can be run multiple times on the same computer, so that on a dual core I can run two tournaments simultaneously. (I am not sure if he now included this in the standard version on his website.)

So a good way to start would be to download PSWBTM, and make as many copies of the folder you install it in as you have cores on your machine. Then download the WinBoard_F executable, and put it in its own folder. You will have to tell PSWBTM (through the tools->configure menu) where the winboard.exe is, but they can share the single WinBoard. You will have to run WinBoard once from its own folder to let it create a winboard.ini file with settings (and be sure to select board size Bulky or Middling if you want to play games involving fairy pieces).

Each PSWBTM will keep his own copy of the winboard.ini file with WinBoard settings, so you would have to copy the winboard.ini that contains the desired settings to all the PSWBTM folders. The only tricky thing is that WinBoard does not save the variant it is playing in the winboard.ini file, but it does read it from the winboard.ini file (and PSWBTM sees to it that during a tournament, the winboard.ini is not overwritten). So you would have to add a line

/variant=capablanca

(or whatever other variant you will be playing the tournament for) in the winboard.ini file with a text editor, or it would just play normal Chess.

I think that is about it. Note that for 10x8 Chess, ChessGUI by Mathias Gemuh is an alternative GUI to WinBoard, and is able to manage tournaments without an external manager. I haven't used it much, though, and I wouldn't know about features for starting from FEN or PGN files.

[Spock]

Well, that truly exceeds my highest hopes, to get a reaction from a person with your experience! Of course I am willing to provide every guidance you might need.

Thanks. My experience is pretty much limited to standard chess with UCI engines under Shredder GUI and chessbase GUIs, as well as FRC under Shedder GUI. Although I've have recently tried Arena as well. Guidance on winboard-F and PSWBTM will be required for sure. I'll download them over the weekend and take a look

First of all, what do you think of my plan, of playing from differently shuffled opening positions, rather than using an external book as CCRL does? I am open to any suggestions.

I think this is the best idea. There are presumably no suitable books anyway that include the chancellor and archbishop ?

If it all works, I would plan to devote a dual core 2.6GHz Opteron machine to this 24/7, or an Intel Quad at 3.0Ghz. Do you know if PSWBTM and Winboard-F have any issues with Vista X64 ?

Doing blitz first as a proof of concept makes sense. I'm not a fan of repeating time controls these days (I quite like 60'+15" and 90'+30" for example), but 40/40 for would be fine for this - I understand that the games are generally shorter anyway

[Matthias Gemuh]

I had missed the special castling squares for Janus chess and Embassy chess, so BigLion80 and ArcBishop80 and ChessGUI have been updated.

Matthias.

[Matthias Gemuh]

I think that is about it. Note that for 10x8 Chess, ChessGUI by Mathias Gemuh is an alternative GUI to WinBoard, and is able to manage tournaments without an external manager. I haven't used it much, though, and I wouldn't know about features for starting from FEN or PGN files.

Starting from EPD or PGN files should work.
Someone would need to test if all FENs are iterated through correctly.

Matthias.

[smrf]

I still have not succeeded in having a single game done at your ChessGUI. It is spoken from tournaments, but I want only to do a single game first. And I see no chances for to select any appropriate timing.

[Matthias Gemuh]

I still have not succeeded in having a single game done at your ChessGUI. It is spoken from tournaments, but I want only to do a single game first. And I see no chances for to select any appropriate timing.

The readme needs to be updated indeed ;(
Maybe I will fix that soon.

Matthias.