Chess variant tournament: Berolina Chess

[enhorning]

Same hardware as usual. Single-thread, on an i-970, pondering off, 512 MB hash, 40 moves in 20 minutes (30 seconds / move average).

Preliminary participants:
Nebiyu 1.43
Sugod 1.0
Sjaak 4.70
SamChess BE 1.0
Fairy-Max 4.8

I haven't worked out how many cycles yet - but it will, much like the Crazyhouse tournament, be multiple from different starting moves.

... I had hoped to include ChessV as well, but did not manage to get it running for Berolina under Winboard.

Next variant tournament after this will be Spartan Chess, and following that, possibly Shatranj.

Result of the test / warm-up blitz tournament:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Su Ne Sa Sj Fa
 1. Sugod v1.0                ## 01 10 11 11   75%   6.0 ( 28.0,  18.0)
 2. NebiyuChess_1.43          10 ## 11 01 11   75%   6.0 ( 28.0,  18.0)
 3. SamChess-BE 1.0           01 00 ## 10 11   50%   4.0 ( 32.0,  10.0)
 4. Sjaak r470                00 10 01 ## 11   50%   4.0 ( 32.0,  10.0)
 5. Fairy-Max 4.8Rt           00 00 00 00 ##    0%   0.0 ( 40.0,   0.0)

So, seem to be quite a close field, except for Fairy-Max.

Tournament will start in a few days, or a week from now... depends on when I and the cpu are available.

[enhorning]

I've had a look at what the different engines suggest for an opening move, and am going to run the tournament with this selection of opening moves:

1. d2-f4 (Sugod's and Fairy-max's preferred move after long thinks)
1. d2-b4 (Sjaak's preferred move after a long think)
1. g2-e4 (SamChess's preferred move... from 1 ply and until the end)
1. a2-b3 (Nebiyu's preferred move after a long think)
1. Nc3 (Nebiyu, Sjaak and Fairy all had this as their choice of move at reasonable depths... even though they later changed their minds)

That will be 10 games for each mini-match, and 100 games total for the tournament. Starting the tournament now.

(Other opening moves that appeared at reasonable depths for various programs included a2-c4, h2-g3, d2-e3, Nf3, b2-d4. Also interesting to observe that SamChess and Sugod seemed to be deterministic, whereas the other three programs gave different moves at various ply counts when run again from a clean start.)

[enhorning]

Standings after 2 cycles out of 10:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Su Ne Sj Fa Sa
 1. Sugod v1.0                ## 11 11 11 11  100%   8.0 ( 24.0,  24.0)
 2. NebiyuChess_1.43          00 ## 11 11 11   75%   6.0 ( 28.0,  12.0)
 3. Sjaak r470                00 00 ## 01 =1   31%   2.5 ( 35.0,   4.3)
 4. Fairy-Max 4.8Rt           00 00 10 ## 10   25%   2.0 ( 36.0,   4.0)
 5. SamChess-BE 1.0           00 00 =0 01 ##   19%   1.5 ( 37.0,   3.3)

So far, Sugod is winning everything, and Nebiyu is beating everybody except Sugod. Let's see if that will hold up through the different openings.

I've also seen some drastic evaluation changes in endgames when programs suddenly realize it's worth sacrificing to get a pawn promotion... pawns are easier to advance, and harder to block.

[enhorning]

Standings after 4 cycles out of 10:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Sugo Nebi Sjaa SamC Fair
 1. Sugod v1.0                #### 1111 1111 1111 1111  100%  16.0 ( 96.0,  96.0)
 2. NebiyuChess_1.43          0000 #### 1111 1110 1111   69%  11.0 (116.0,  47.5)
 3. Sjaak r470                0000 0000 #### =101 0111   34%   5.5 (138.0,  20.3)
 4. SamChess-BE 1.0           0000 0001 =010 #### 0110   28%   4.5 (142.0,  25.3)
 5. Fairy-Max 4.8Rt           0000 0000 1000 1001 ####   19%   3.0 (148.0,  14.5)

Sugod is still winning every game, but Nebiyu had it's first loss against one of the lower three programs.

Taking advantage of Winboard's concurrency currently, as I have nothing else running on the computer, so playing 4 tournament games in parallel.

[enhorning]

Standings after 6 cycles out of 10:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Sug Neb Sja Sam Fai
 1. Sugod v1.0                ### 111 111 111 111
                              ### 111 111 111 111  100%  24.0 (216.0, 216.0)

 2. NebiyuChess_1.43          000 ### 111 111 111
                              000 ### 111 011 111   71%  17.0 (258.0, 106.5)

 3. Sjaak r470                000 000 ### =10 011
                              000 000 ### 101 111   35%   8.5 (309.0,  41.3)

 4. SamChess-BE 1.0           000 000 =01 ### 011
                              000 100 010 ### 011   31%   7.5 (315.0,  50.3)

 5. Fairy-Max 4.8Rt           000 000 100 100 ###
                              000 000 000 100 ###   12%   3.0 (342.0,  23.5)

[enhorning]

Congratulations to Sugod - a dominating performance, even though it did not manage to remain undefeated.

Final standings:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Sugod Nebiy SamCh Sjaak Fairy
 1. Sugod v1.0                ##### 11111 11111 11111 11111
                              ##### 11011 11111 1111= 11111   96%  38.5 (605.0, 570.8)

 2. NebiyuChess_1.43          00000 ##### 11101 11111 11111
                              00100 ##### 11110 11111 11111   72%  29.0 (700.0, 326.5)

 3. SamChess-BE 1.0           00000 00010 ##### =0101 01101
                              00000 00001 ##### 01111 1010    35%  13.5 (847.5, 163.8)

 4. Sjaak r470                00000 00000 =1010 ##### 01111
                              0000= 00000 10000 ##### 110=0   26%  10.5 (885.0, 115.3)

 5. Fairy-Max 4.8Rt           00000 00000 10010 10000 #####
                              00000 00000 0101  001=1 #####   19%   7.5 (901.5,  90.8)

Here is an example win of Sugod over Nebiyu:

Code: Select all

[Event "Computer Chess Game"]
[Site "ENHORNING"]
[Date "2013.01.18"]
[Round "16"]
[White "NebiyuChess_1.43"]
[Black "Sugod v1.0"]
[Result "0-1"]
[TimeControl "40/1200"]
[Variant "berolina"]
[Annotator "2. +0.05   1... -0.31"]

1. d2b4 a7b6 {-0.31/17 1:00} 2. h2f4 {+0.05/20 29} e7c5 {-0.22/17 53} 3.
Nf3 {+0.05/19 29} Nc6 {-0.03/16 28} 4. c2b3 {+0.00/19 22} Bd6 {+0.06/17 30}
5. Qd3 {+0.00/19 30} Nf6 {+0.06/16 51} 6. Nc3 {+0.05/20 30} O-O
{+0.00/17 34} 7. f2d4 {+0.05/19 30} f7g6 {+0.85/16 35} 8. Bd2 {-0.05/18 27}
Qe8 {+1.26/16 42} 9. Qf5 {-0.05/18 24} Na5 {+1.61/16 26} 10. Qd3
{-0.35/18 30} Nc4 {+1.80/16 24} 11. Bc1 {-0.55/19 26} d7b5 {+1.87/16 21}
12. Rh4 {-0.45/17 37} Bd7 {+2.27/15 40} 13. b4xb5 {-0.60/17 23} b6xb5
{+2.45/14 4} 14. b2a3 {-0.95/19 30} Bc6 {+3.23/15 37} 15. a3b4
{-1.00/19 30} Be5 {+3.36/15 26} 16. Nd2 {-1.15/19 30} Nxd2 {+3.12/16 27}
17. Bxd2 {-1.65/20 30} g6h5 {+3.32/16 33} 18. Rh3 {-1.45/20 26} Rd8
{+3.33/16 31} 19. Be3 {-1.40/19 28} Ng4 {+3.15/17 30} 20. Nxb5
{-1.60/18 28} Bxf4 {+3.94/16 26} 21. Bxf4 {-1.90/19 41} Rxf4 {+3.84/17 7}
22. g2f3 {-2.65/21 45} Bxb5 {+7.57/19 27} 23. b4xb5 {-4.00/24 23} Rfxd4
{+11.24/19 27} 24. Qc3 {-7.70/25 40} Rd3 {+12.46/20 24} 25. Qxd3
{-8.90/24 24} Rxd3 {+13.07/20 8} 26. f3e4 {-10.10/26 26} Qxb5
{+13.74/19 28} 27. Rxd3 {-10.75/27 27} Qxd3 {+13.97/17 12} 28. Rd1
{-10.60/26 32} Qg3+ {+14.72/17 29} 29. Kd2 Nf2 {+15.53/15 28} 30. Kc2
{-11.90/25 35} Nxd1 {+22.34/12 25} 31. Kxd1 {-12.05/24 35} c5d4
{+24.62/13 30} 32. e2f3 {-299.81/23 35} d4c3 {+319.85/14 28} 33. Bc4+
{-299.85/24 35} Kh8 {+319.87/13 30} 34. Bf1 {-299.87/23 35} c3b2
{+319.89/11 30} 35. Kc2 {-299.89/22 36} Qf2+ {+319.91/10 30} 36. Be2
{-299.91/21 36} Qxe2+ {+319.93/5 0.1} 37. Kc3 {-299.93/19 36} b2c1=Q+
{+319.95/5 0.1} 38. Kd4 {-299.95/18 29} Qeb2+ {+319.97/5 0.1} 39. Kd3
{-299.97/18 33} Qcc3# {+319.99/5 0.1}
{Black mates} 0-1

And here is the single win by Nebiyu over Sugod:

Code: Select all

[Event "Computer Chess Game"]
[Site "ENHORNING"]
[Date "2013.01.19"]
[Round "36"]
[White "NebiyuChess_1.43"]
[Black "Sugod v1.0"]
[Result "1-0"]
[TimeControl "40/1200"]
[Variant "berolina"]
[Annotator "2. +0.15   1... -0.34"]

1. a2b3 h7g6 {-0.34/17 1:00} 2. Nf3 {+0.15/17 24} c7d6 {-0.21/16 57} 3.
b2d4 {+0.20/19 22} Nc6 {-0.12/16 28} 4. c2e4 {+0.25/19 30} Qb6
{-0.26/15 37} 5. d2b4 {+0.30/17 30} Nf6 {-0.36/16 37} 6. Nc3 {+0.55/18 23}
Rh5 {-0.48/16 49} 7. e2c4 {+0.70/17 24} Na5 {-1.47/14 47} 8. Bd2
{+0.90/16 24} Qc7 {-1.39/14 44} 9. f2h4 {+1.05/19 30} Rh8 {-1.71/17 31} 10.
b4c5 {+1.05/19 30} b7a6 {-1.79/17 31} 11. e4f5 {+1.10/20 27} Ng8
{-1.85/18 39} 12. Bd3 {+1.20/18 31} Nh6 {-1.84/15 37} 13. h2f4
{+1.25/18 30} Rb8 {-1.57/16 35} 14. Rb1 {+1.25/20 29} Ng4 {-1.48/16 33} 15.
Rh3 {+1.15/19 31} Bb7 {-1.25/16 32} 16. Nb5 {+1.70/21 26} Qc8 {-1.37/16 30}
17. b3a4 {+1.65/20 31} a7b6 {-1.16/16 19} 18. Nxd6+ {+1.70/21 31} d7xd6
{-1.46/16 4} 19. a4xa5 {+1.40/21 31} a6b5 {-1.45/16 27} 20. Qe2
{+1.55/21 31} f7e6 {-1.53/16 26} 21. Rb3 {+1.60/20 31} Qd7 {-1.19/15 25}
22. Kd1 {+1.60/20 31} Kf7 {-1.32/15 24} 23. f4g5 {+1.90/20 32} Rd8
{-1.40/14 23} 24. Rg3 {+1.95/21 32} Bxf3 {-1.67/16 23} 25. g5f6
{+1.95/23 32} Kg8 {-1.48/19 23} 26. Qxf3 {+1.95/25 32} Ne5 {-1.57/18 23}
27. Qf4 {+1.95/25 30} Nxd3 {-1.66/17 23} 28. Rbxd3 {+1.95/24 26} d6e5
{-1.70/16 23} 29. Ke2 {+1.90/22 33} Qe8 {-1.79/16 23} 30. g2e4
{+2.30/21 27} e5xe4 {-1.80/16 23} 31. c5d6 {+2.25/20 34} Qc6 {-2.23/15 23}
32. d4c5 {+2.40/23 32} Qc7 {-2.82/17 23} 33. Kf3 {+2.40/24 27} b5a4+
{-1.95/17 23} 34. Kxe4 {+2.65/21 36} Rh7 {-1.61/16 23} 35. Rg1
{+3.15/20 36} a4b3 {-2.07/16 24} 36. Bb4 {+3.20/21 33} e6d5+ {-2.46/16 24}
37. Rxd5 {+4.25/19 30} Qb7 {-2.73/16 24} 38. Kd4 {+4.20/19 32} Kh8
{-2.90/16 24} 39. Rb1 {+4.25/22 36} b3c2 {-3.01/17 24} 40. Be1
{+4.30/22 48} g6h5 {-3.07/16 31} 41. h4g5 {+4.55/19 22} Rh6 {-3.67/16 1:01}
42. Ra1 {+4.55/20 22} h5g4 {-2.90/15 58} 43. Qxg4 {+5.20/17 22} Qb8
{-4.25/15 55} 44. Bg3 {+5.55/21 28} Kg8 {-5.40/15 52} 45. Qe2 {+5.50/21 30}
Rc8 {-5.37/16 49} 46. Qxc2 {+5.65/20 25} Rh3 {-7.15/17 47} 47. Qg2
{+5.80/20 30} Rh6 {-8.02/17 45} 48. Bf4 {+6.50/21 25} Rc7 {-8.54/16 42} 49.
f5g6 {+7.20/21 30} Qc8 {-10.05/17 40} 50. Re5 {+9.95/24 30} Qb7
{-11.49/18 38} 51. Qxb7 {+10.80/24 30} Rxb7 {-11.49/18 9} 52. g6f7+
{+12.30/26 30} Rb8 {-12.67/18 36} 53. Rf5+ {+12.65/26 28} Rd8
{-14.52/18 23} 54. Rb1+ {+12.65/25 30} Rh3 {-16.16/17 33} 55. Rxb6+
{+15.85/24 25} Ra3 {-17.78/17 31} 56. Rd5+ {+22.50/27 31} Ra1
{-21.74/15 23} 57. d6c7+ {+27.55/26 28} Rd1+ {-40.31/19 29} 58. Ke4+
{+299.78/27 31} R1xd5 {-39.66/16 27} 59. c7b8=Q+ {+299.82/27 31} Rd4+
{-319.78/19 24} 60. Kf5+ {+299.84/27 31} R4d5+ {-319.80/17 25} 61. Kg4+
{+299.86/25 31} Rd1 {-319.82/17 23} 62. c5d6+ {+299.88/25 31} g7h6
{-319.90/13 22} 63. Qxd8+ {+299.90/24 30} Kh7 {-319.92/13 22} 64. f7xf8=R
{+299.92/19 31} Rg1+ {-319.94/5 0.1} 65. Bg3 {+299.94/19 31} Rxg3+
{-319.96/5 0.1} 66. Kxg3 {+299.96/18 8} Kg7 {-319.98/5 0.1} 67. Rf7+
{+299.98/18 33}
{White wins} 1-0

As usual, if somebody wants the full PGN of all games, either PM me or email me at enhorning@gmail.com .

Next tournament, in roughly two weeks time, will be Spartan chess, and my preliminary list of programs are these:
Catalyst
Fairy-Max
Nebiyu
Sangga (Deuterium)
Sjaak

If anybody knows of other Spartan chess programs that are available, please let me know. I will be curious what these games turn out like, as it's not a variant I am personally familiar with. Once the time draws closer, that tournament will get its own thread.

After that, Shatranj is probably next, and I am open to suggestions for variants to run in the future (as long as there exist sufficient number of Winboard-compatible engines for them).

[enhorning]

Hmm... I just noticed that there is a game missing from the PGN-file (and cross-table) - SamChess - FairyMax opening with 1. a2-b3.

I wonder what happened to that - looking at the Winboard .trn file, it does indeed list 100 results:

Code: Select all

-results "=+---+-+--+-+-+-+-+-++-+-+-+-++-+-+++-++++-+-+-+-++-+-+-+-+-++-+-+-+-+-++++-+-++++-=-+-+-+--+++++=++"

Is there a way to work out which of these results belong to the missing game?

I wonder why that game did not get written to the pgn file - could this be due to something not working with concurrency? - that game took place during the time I was running 4 games in parallel.

Oh wait - that must have been the game where FairyMax suffered a bug, and refused to accept a valid move. I wonder why that didn't get written to the PGN - I did adjudicate it (as I had to do with a few other games that for some reason didn't move on to the next game after checkmate occurred). Luckily, I sent HGM an email about the bug, so have a copy of the game there.

So, after including that game in the PGN file, the corrected cross-table is:

Code: Select all

Cross table, sorted by score percentage, Buchholz, SB

                              Sugod Nebiy SamCh Sjaak Fairy
 1. Sugod v1.0                ##### 11111 11111 11111 11111
                              ##### 11011 11111 1111= 11111   96%  38.5 (615.0, 580.8)

 2. NebiyuChess_1.43          00000 ##### 11101 11111 11111
                              00100 ##### 11110 11111 11111   72%  29.0 (710.0, 334.5)

 3. SamChess-BE 1.0           00000 00010 ##### =0101 01101
                              00000 00001 ##### 01111 10101   36%  14.5 (855.0, 171.3)

 4. Sjaak r470                00000 00000 =1010 ##### 01111
                              0000= 00000 10000 ##### 110=0   26%  10.5 (895.0, 118.8)

 5. Fairy-Max 4.8Rt           00000 00000 10010 10000 #####
                              00000 00000 01010 001=1 #####   19%   7.5 (925.0,  94.8)

[Ferdy]

Thanks for the tourney, you may send the games to the same email you sent the zh games.
BTW I am developing a shatranj variant and hope the tourney will happen .

[enhorning]

Shatranj definitely will happen at some point, as I quite enjoy Shatranj (despite drawishness) and there are a reasonable number of programs that support it! Just a question of when I will get around to it (and if somebody tempts me to run something else inbetween).

[Evert]

Next tournament, in roughly two weeks time, will be Spartan chess, and my preliminary list of programs are these:
Catalyst
Fairy-Max
Nebiyu
Sangga (Deuterium)
Sjaak

If anybody knows of other Spartan chess programs that are available, please let me know. I will be curious what these games turn out like, as it's not a variant I am personally familiar with. Once the time draws closer, that tournament will get its own thread.

After that, Shatranj is probably next, and I am open to suggestions for variants to run in the future (as long as there exist sufficient number of Winboard-compatible engines for them).

Yikes, I need to pay more attention to the Tournaments board!

Nice to see some variant tournaments, variants don't get enough love (except for Fischer, which as variants go I don't find particularly interesting). Anyway, to address the above question: I have a second dedicated (ish) variant program called Leonidas that plays Spartan chess (as an extra it also plays regular chess and at some point it will also play Berolina chess and Seirawan chess). It will hopefully play very different from Sjaak (and be stronger), but it's not quite there yet. Let me know if you're interested (I can understand if you're not, since you already have Sjaak in there). There's also Spartacus by Harm Geert Muller, but I think that's private, I don't know if that's a concern.

I may have some time to get back to working on Leonidas. I got side-tracked from working on it because I needed a (command-line, ie, non-GUI) referee program for tuning, but I have one now (based on Sjaak) so that's no longer a concern.