Chess variant tournament: Capablanca and similar

[enhorning]

Btw, Martin Sedlak's engine Heretic is still missing from your list.

Thanks, downloaded that.

Adding that to the tournament, as well as the Spartacus and TSCPGothic you sent me, we now have 13 participants:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Bi Ne Sp TJ Jo Fa Sm Sj TS He Pa Bi Ar
 1. Bihasa v3.2               ## 11 10 01 11 11 11 11 11 11 11 11 11   92%  22.0 (268.0, 227.5)
 2. NebiyuChess_1.43          00 ## 11 01 11 11 11 11 11 11 11 11 11   87%  21.0 (270.0, 206.5)
 3. Spartacus 1.02            01 00 ## 11 11 11 11 11 11 11 11 11 11   87%  21.0 (270.0, 206.0)
 4. TJchess10x8 1.1-x64       10 10 00 ## 11 =1 11 11 11 11 11 11 11   81%  19.5 (273.0, 182.0)
 5. Joker80.np                00 00 00 00 ## 1= 11 11 0= 01 01 11 11   50%  12.0 (288.0,  86.3)
 6. Fairy-Max 4.8S            00 00 00 =0 0= ## 10 01 11 01 11 11 11   50%  12.0 (288.0,  84.3)
 7. Smirf BC-173g-X           00 00 00 00 00 01 ## 01 01 11 11 11 11   46%  11.0 (290.0,  69.5)
 8. Sjaak 524                 00 00 00 00 00 10 10 ## 00 11 01 11 11   37%   9.0 (294.0,  55.5)
 9. TSCPtest                  00 00 00 00 1= 00 10 11 ## 10 00 11 01   35%   8.5 (295.0,  62.5)
10. Heretic 0.3               00 00 00 00 10 10 00 00 01 ## =1 11 11   35%   8.5 (295.0,  51.8)
11. Pair-o-Max 4.8S           00 00 00 00 10 00 00 10 11 =0 ## 10 11   31%   7.5 (297.0,  47.3)
12. BigLion80 2.23x WB        00 00 00 00 00 00 00 00 00 00 01 ## 11   12%   3.0 (306.0,   9.5)
13. ArcBishop80 1.01 WB       00 00 00 00 00 00 00 00 10 00 00 00 ##    4%   1.0 (310.0,   8.5)

... judging from the blitz (40 moves / 1 minute) test results (with weird back row), we should get quite a fight at the top - 4 programs all fairly close to each other! Quite a close group in the middle as well.

[hgm]

I think the only thing that has to be in the ChessV SGF file is the parent variant (Capablanca Chess) and the setup. Like

Code: Select all

Gothic Chess:Capablenca Chess
$array="rnbqckabnr/pppppppppp/10/10/10/10/PPPPPPPPPP/RNBQCKABNR"

Put this in a file Gothic.sgf, and then install ChessV with the engine command "ChessV_WinBoard.exe Gothic.sgf". (I.e. specify Gothic.sgf as engine parameter.)

[enhorning]

Thanks, that seems to work (so far, at least - partway through one game) - if I encounter no trouble running it through the gauntlet, we'll have 14 participants then!

Oh - it crashes at end of first / start of second game. Hmm...

*looks at the line I used for it for Shatranj*

Okay, added -xreuse and -features ping (or something like that).

Starting Gauntlet again, and it went on to second game fine this time - so good hopes that it will work all the way through.

If anybody wants me to use a newer version of their program, or one tuned more for Capa variants, I need to have it within 24 hours - I plan to start the tourney around this time tomorrow morning!

[hgm]

OK, I guess the -xreuse is always important for ChessV. It cannot play two games without restarting the engine.

IIRC the -firstFeatures ping=1 was only needed because we had to install ChessV as a WB v1 engine to make WB ignore the list of variants it says it can play, because shatranj was missing from that. But running as WB v1 engine WB would also not know that the engine supports ping, so we needed to tell it that with a -firstFeatures command.

For variant capablanca it should not be needed to install ChessV as WB v1, as capablanca is amongst the supported variants it reports. So then you also would not need the -firstFeatures option, because as WB v2 engine ChessV will send this to WB by itself. (It should also be considered a ChessV bug that it does not send reuse=0 with its features, so that you need the -xreuse.) It will not hurt to have it, though.

[enhorning]

Okay, ChessV worked fine in its blitz gauntlet:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Bi Ne Sp TJ Jo Fa Sm TS He Sj Pa Ch Bi Ar
 1. Bihasa v3.2               ## 11 10 01 11 11 11 11 11 11 11 11 11 11   92%  24.0 (316.0, 271.5)
 2. NebiyuChess_1.43          00 ## 11 01 11 11 11 11 11 11 11 11 11 11   88%  23.0 (318.0, 248.5)
 3. Spartacus 1.02            01 00 ## 11 11 11 11 11 11 11 11 11 11 11   88%  23.0 (318.0, 248.0)
 4. TJchess10x8 1.1-x64       10 10 00 ## 11 =1 11 11 11 11 11 11 11 11   83%  21.5 (321.0, 221.5)
 5. Joker80.np                00 00 00 00 ## 1= 11 0= 01 11 01 11 11 11   54%  14.0 (336.0, 110.8)
 6. Fairy-Max 4.8S            00 00 00 =0 0= ## 10 11 01 01 11 01 11 11   50%  13.0 (338.0, 106.3)
 7. Smirf BC-173g-X           00 00 00 00 00 01 ## 01 11 01 11 =1 11 11   48%  12.5 (339.0,  91.8)
 8. TSCPtest                  00 00 00 00 1= 00 10 ## 10 11 00 11 11 01   40%  10.5 (343.0,  83.0)
 9. Heretic 0.3               00 00 00 00 10 10 00 01 ## 00 =1 11 11 11   40%  10.5 (343.0,  72.8)
10. Sjaak 524                 00 00 00 00 00 10 10 00 11 ## 01 01 11 11   38%  10.0 (344.0,  71.5)
11. Pair-o-Max 4.8S           00 00 00 00 10 00 00 11 =0 10 ## 11 10 11   37%   9.5 (345.0,  68.3)
12. ChessV_WinBoard           00 00 00 00 00 10 =0 00 00 10 00 ## 11 10   21%   5.5 (353.0,  37.3)
13. BigLion80 2.23x WB        00 00 00 00 00 00 00 00 00 00 01 00 ## 11   12%   3.0 (358.0,  13.5)
14. ArcBishop80 1.01 WB       00 00 00 00 00 00 00 10 00 00 00 01 00 ##    8%   2.0 (360.0,  16.0)

... so it's in for the main tourney as well!

[hgm]

I managed to get the Pair-o-Max re-write running! I uploaded it to

http://hgm.nubati.net/MaxPlus.exe

would like this to replace Pair-o-Max in the tourney. You can place it in the same folder as the old pairomax.exe; it uses the same fmax.ini file from that folder.

Compared to the old Pair-o-Max / Fairy-Max, it has:
*) Completely re-written AI routine, using verbose variable names, so that it has a much more readable source.
*) Stores the complete list of moves, and then plays them from the list, rather than always playing them in move-generation order.
*) Any hash move will be played first. (Fairy-Max excluded castlings and Pawn double-pushes, although in those cases it started with moves of the piece that made the hash move.)
*) Sorts the captures MVV/LVA before playing them.
*) Plays a killer move before the rest of the non-captures.
*) Sorts any move that scores above alpha to the front of the move list.
*) Recognizes repetitions of search positions rather than just from the game history. (Not sure if this is beneficial, though.)
*) Uses "shallowest-of-two" hash replacement (+undercut) in stead of "always replace".

It seems a lot better than the old Pair-o-Max, and dramatically so in Capablanca Chess! Initial results (new vs old version):
normal Chess: 38+ 9- 23= (70%)
Capablanca; 34+ 3- 11= (82%)

OK, not that many games yet, but 34 vs 3 seems significant...

[enhorning]

I managed to get the Pair-o-Max re-write running! I uploaded it to

http://hgm.nubati.net/MaxPlus.exe

would like this to replace Pair-o-Max in the tourney. You can place it in the same folder as the old pairomax.exe; it uses the same fmax.ini file from that folder.

Okay, will replace Pair-o-Max with this.

From running my blitz gauntlet, it seems a decent improvement, although my results were not as dramatical as your selfplay results:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Bi Ne Sp TJ Fa Ma Jo Sm TS He Pa Sj Ch Bi Ar
 1. Bihasa v3.2               ## 11 10 01 11 11 11 11 11 11 11 11 11 11 11   93%  26.0 (368.0, 319.5)
 2. NebiyuChess_1.43          00 ## 11 01 11 11 11 11 11 11 11 11 11 11 11   89%  25.0 (370.0, 294.5)
 3. Spartacus 1.02            01 00 ## 11 11 11 11 11 11 11 11 11 11 11 11   89%  25.0 (370.0, 294.0)
 4. TJchess10x8 1.1-x64       10 10 00 ## =1 11 11 11 11 11 11 11 11 11 11   84%  23.5 (373.0, 265.0)
 5. Fairy-Max 4.8S            00 00 00 =0 ## == 0= 10 11 01 11 01 01 11 11   50%  14.0 (392.0, 128.8)
 6. Max-Plus 4.8S             00 00 00 00 == ## 11 01 01 10 0= =1 11 11 11   50%  14.0 (392.0, 120.8)
 7. Joker80.np                00 00 00 00 1= 00 ## 11 0= 01 01 11 11 11 11   50%  14.0 (392.0, 118.3)
 8. Smirf BC-173g-X           00 00 00 00 01 10 00 ## 01 11 11 01 =1 11 11   48%  13.5 (393.0, 113.3)
 9. TSCPtest                  00 00 00 00 00 10 1= 10 ## 10 00 11 11 11 01   41%  11.5 (397.0, 100.0)
10. Heretic 0.3               00 00 00 00 10 01 10 00 01 ## =1 00 11 11 11   41%  11.5 (397.0,  91.0)
11. Pair-o-Max 4.8S           00 00 00 00 00 1= 10 00 11 =0 ## 10 11 10 11   39%  11.0 (398.0,  92.3)
12. Sjaak 524                 00 00 00 00 10 =0 00 10 00 11 01 ## 01 11 11   37%  10.5 (399.0,  84.0)
13. ChessV_WinBoard           00 00 00 00 10 00 00 =0 00 00 00 10 ## 11 10   20%   5.5 (409.0,  39.3)
14. BigLion80 2.23x WB        00 00 00 00 00 00 00 00 00 00 01 00 00 ## 11   11%   3.0 (414.0,  15.0)
15. ArcBishop80 1.01 WB       00 00 00 00 00 00 00 00 10 00 00 00 01 00 ##    7%   2.0 (416.0,  17.0)

[hgm]

Well, your blitz gauntlet has very few games (per engine), and the ranking you get is highly suspect. Fairy-Max has ended way higher than it is supposed to be: it is typically some 400 Elo weaker than Joker80, and even weaker than TSCP. I don't expect the changes could close that gap of 400 Elo. Compared to Joker the new search is still very basic (only a single killer in stead of two, 2 hash entries per bucket in stead of 7, no SEE to weed out bad captures, no special check test, so that checks are only detected one ply later), and the evaluation still is identical to the 'nothing of anything' evaluation of Fairy-Max, where Joker has elaborate King safety, Rook open files/7th-rank bonus, passer recognition, backward, isolated, doubled Pawn scoring. SO it must be a lot weaker than Joker.

Sjaak should be stronger than Fairy-Max.

Of course self-play is known to exaggerate rating differences. (But therefore a sensitive method to measure improvements.)

[enhorning]

Well, your blitz gauntlet has very few games (per engine), and the ranking you get is highly suspect.

Of course!

The main point of it is to weed out technical troubles.

[Daniel Shawul]

I have a new version here https://sites.google.com/site/dshawul/N ... ects=0&d=1
I did not work on the evaluation but only some changes in the search. Bihasa is just too strong for nebiyu right now. Nebiyu has a hard time getting a win out of 10 games. I have to think of cheaper ways to include some king safety terms because that seems to be its major problem.