Seirawan Chess: Sigla, Sjaak, Leonidas

[hgm]

I tried a few games of Seirawan Chess with the various engines that play it. And from the face of it there seems to be something very wrong with these engines: they seem to make incredibly stupid blunders in the opening. Sigla, for instance, plays:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.14"]
[Round "1"]
[White "Sigla v1.0"]
[Black "S-Fruit"]
[Result "0-1"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nc3/H {+0.00/10} Nf6 {-0.19/10 1.6} 2. g4 {+0.46/10 1.4} Nxg4
{+0.43/9 1.8} 3. Nh3/E {+1.70/8 1.4} d5 {+0.61/9 1.9} 4. f3 {+0.39/8 1.4}
Nf6 {+0.89/9 1.4} 5. Nf4 {-0.49/8 1.4} e5 {+1.80/9 1.3} 6. Nfxd5
{+0.76/7 1.4} Nxd5 {+2.79/7 0.7} 7. Nxd5 {-0.30/8 1.4} Qh4/E+ {+2.96/8 1.4}
8. Eg3 {-2.48/10 1.4} Exd5 {+2.96/8 1.6} ...

It seems to sac g4 for no purpose (although S-Fruit does not get very enthousiastic about that yet), and 6. Nfxd5 just blunders away a piece by capturing a Pawn that is sufficiently protected.

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.14"]
[Round "3"]
[White "Sjaak II unknown revision "]
[Black "S-Fruit"]
[Result "0-1"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nf3/E {+0.09/8} Nc6 {-0.26/10 1.1} 2. Nc3/H {+0.64/8 1.4} Nf6
{-0.06/10 1.6} 3. e4 {+2.70/8 1.5} d5 {-0.23/9 1.1} 4. Nxd5 {+4.97/5 1.5}
Nxe4 {-0.05/9 1.7} 5. Nb4 {+3.45/5 0.8} Nxb4 {+3.03/9 2.5}...

5. Nb4 seems to blunder away a Knight by moving it unprotected to an attacked square. If I let it think for the opponent, it eventually does find the Knight is up for grabs, but it takes amazingly long for that, and at 7 ply it still thinks it can gain it back through 7. Qa4+ 8. Qxb2, overlooking the interposition evasion 7... Nc6:

Code: Select all

dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 11	+2.76 	5.89M  	1:11.34	6. ... Nxb4 7. Hc3 Q8d6/E 8. Hb5 B8d7/H 9. Hxd6 Hxd6 10.  d3 Nxa2 11.  h3 Ee6 12. Ee2 Nb4
 10	+2.60 	3.25M  	0:38.98	6. ... Nxb4 7. Hc3 Q8d6/E 8. Qb3  a5 9. Hb5 B8d7/H 10. Hxd6 Hxd6 11.  d3 Hc5 12. Qc3 Nxa2
  9	+1.52 	2.10M  	0:25.04	6. ... Nxb4 7. Qa4 Nc6 8.  d4 B8f5/H 9. Bd3 Q8d7/E 10. Bxf5 Qxf5 11. Be3
  8	+0.78 	449540	0:05.06	6. ... Nxb4 7. Qa4 Q8d7/E 8. Qxb4  e6 9. Qc3 B8c5/H 10.  d3 Qa4
  7	+0.34 	274607	0:03.06	6. ... Nxb4 7. Qa4 Q8d7/E 8. Qxb4 Ee6 9. Ee2  c5 10. Qc3 R8b8/H
  6	 -0.24 	119468	0:01.26	6. ... Nxb4 7.  a3 Q8d6/E 8.  c5 Qxc5 9.  axb4 Qxc1
  5	  0.00 	71421  	0:00.73	6. ... Nxb4 7. Qa4 B8d7/E 8. Qxb4 R8b8/H
  4	 -1.86 	25286  	0:00.26	6. ... Nxb4 7.  d3 B8e6/H 8. Qd2
  3	 -3.43 	8956    	0:00.09	6. ... Nxb4 7. Qa4 Nc6
  2	 -5.06 	5156    	0:00.04	6. ... B8d7/E 7. Hc3

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.14"]
[Round "2"]
[White "S-Fruit"]
[Black "Sjaak II unknown revision "]
[Result "1-0"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nc3 {+0.16/10} Nc6/E {+0.34/8 1.6} 2. d4 {+0.69/9 1.8} d6 {-0.61/6 0.9}
3. e4 {+0.88/9 1.6} Be6/H {+0.24/6 1.6} 4. d5 {+3.16/9 1.7} f5
{-0.58/5 2.7} 5. dxe6 {+3.98/7 1.3} Nf6 {-2.74/5 1.7} 6. exf5 {+5.16/8 1.3} ...

Here it bungles a piece by the most simplistic Pawn fork without blinking an eye before the piece is actually gone. From looking at the PV in various positions it seems that it thinks every evasion of either N or B works, because it only considers gating moves for white on ply 2 (but not on other even plies).

Leonidas also seems prone to 'phantom gains' in its evaluation:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.14"]
[Round "3"]
[White "Leonidas 8.3"]
[Black "S-Fruit"]
[Result "0-1"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. d4 {+0.01/9} Nf6 {-0.31/10 2.0} 2. Nc3/E {+0.07/9 1.3} d5 {-0.15/9 1.1}
3. Bf4/H {+0.00/9 1.5} Bf5 {-0.25/9 1.6} 4. Nf3 {+0.17/8 0.8} Nc6
{-0.27/9 1.9} 5. He3 {+2.57/7 0.9} e6 {+0.29/9 1.7} 6. Ed2 {+1.35/6 1.1}
Ng4 {+4.14/9 1.5} 7. Bxc7 {-1.36/8 1.2} Nxe3 {+7.97/8 0.5} ...

The +2.57 on 5. He3 seems totally illusory and evaporates on the next move, where it still dreams of positive score while it steps off the cliff.

[Ferdy]

I tried a few games of Seirawan Chess with the various engines that play it. And from the face of it there seems to be something very wrong with these engines: they seem to make incredibly stupid blunders in the opening. Sigla, for instance, plays:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.14"]
[Round "1"]
[White "Sigla v1.0"]
[Black "S-Fruit"]
[Result "0-1"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nc3/H {+0.00/10} Nf6 {-0.19/10 1.6} 2. g4 {+0.46/10 1.4} Nxg4
{+0.43/9 1.8} 3. Nh3/E {+1.70/8 1.4} d5 {+0.61/9 1.9} 4. f3 {+0.39/8 1.4}
Nf6 {+0.89/9 1.4} 5. Nf4 {-0.49/8 1.4} e5 {+1.80/9 1.3} 6. Nfxd5
{+0.76/7 1.4} Nxd5 {+2.79/7 0.7} 7. Nxd5 {-0.30/8 1.4} Qh4/E+ {+2.96/8 1.4}
8. Eg3 {-2.48/10 1.4} Exd5 {+2.96/8 1.6} ...

It seems to sac g4 for no purpose (although S-Fruit does not get very enthousiastic about that yet), and 6. Nfxd5 just blunders away a piece by capturing a Pawn that is sufficiently protected.

That version does not consider on-hold material in the eval.
I have that fixed already but was not able to issue. I will look at it again and maybe will release a new version within 2 days.

[hgm]

I suspected something like that; it reeked of extreme horizon effect due to pushing a gating over the horizon.

In S-Fruit I keep track incrementally of the number of virgin pieces (as well as their location in the castling flags word). I then use a table HandLoss[NrOfPiecesInHand][NrOfVirginPieces] to correct the evaluation with the value found in this table. This can be used to discount the value of pieces that cannot be gated anymore, and give modest penalties when you start to get perilously close to that.

[Ferdy]

I suspected something like that; it reeked of extreme horizon effect due to pushing a gating over the horizon.

In S-Fruit I keep track incrementally of the number of virgin pieces (as well as their location in the castling flags word). I then use a table HandLoss[NrOfPiecesInHand][NrOfVirginPieces] to correct the evaluation with the value found in this table. This can be used to discount the value of pieces that cannot be gated anymore, and give modest penalties when you start to get perilously close to that.

Version 1.2 will not play g4 I guess though there was g4 in one of the iterations.

Code: Select all

 13	+0.31 	37.1M  	2:17.06	g1f3e b8c6h d2d4 h8g8e a2a3 d7d6 c1f4 a7a6 d4d5 c6a5 d1d4 c8f5 h2h3
 12	+0.17 	7.41M  	0:26.85	g1f3e b8c6h g2g4 e7e6 g4g5 f6h5 e2e3 f8d6e c3e4 d6b4 a2a3 b4e7
 11	+0.37 	3.28M  	0:11.80	g1f3 b8c6h d2d4 e7e6 c1e3e f8e7e d4d5 e6d5 c3d5 f6d5 d1d5 c6b4 d5f5 b4a2
 10	+0.14 	823766	0:02.81	g1f3e b8c6h g2g4 h7h6 h2h3 h8g8e d2d3 g7g5 c1e3 d7d6
  9	+0.38 	223787	0:00.73	d2d4 b8a6h g1f3e d7d6 a2a3 c8d7e c1f4 d7c6 h2h3
  8	+0.32 	72496  	0:00.23	g2g4 b8c6h g4g5 h8g8e g1f3e f6h5 c3d5 d7d6
  7	+0.40 	5063    	0:00.02	g1f3e b8c6h d2d3 h8g8e c1f4 d7d6 h2h3
  6	+0.29 	2559    	0:00.01	g1f3e b8c6h d2d3 h8g8e c1f4 d7d6
  5	+0.47 	1356    	0:00.00	g1f3e b8c6h d2d3 h8g8e c1f4
  4	+0.11 	767      	0:00.00	g1f3e b8c6h d2d3 h8g8e
  3	+0.84 	289      	0:00.00	g1f3e b8c6e d2d3
  2	+0.72 	108      	0:00.00	g1f3e b8c6e
  1	+2.19 	25        	0:00.00	g1f3e

It is good you were able to make the s-fruit worked. We need more engines with this variant , almost every possible piece combinations are here, nice to watch the games.

Just saw the new fen, Sigla does not consider the extra fields for virgin pieces, so it will not work properly when using fen. But it is okay when the game will start from pgn files for example, I tracked it through move history.

[hgm]

Another Sjaak II problem:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.15"]
[Round "10"]
[White "Fairy-Max 4.8V"]
[Black "Sjaak II unknown revision "]
[Result "1-0"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nf3/E {+0.15/6} d6 {-0.16/7 0.8} 2. Nc3/H {+0.00/6 1.1} Nf6/E
{-0.23/7 1.8} 3. Ng5 {-0.19/6 1.2} Nc6/H {+0.08/6 1.8} 4. d3 {+0.07/7 2.2}
e5 {+0.06/6 1.1} 5. Hd2 {+0.10/7 1.2} d5 {+1.15/5 0.9} 6. e4 {-0.16/6 1.9}
h6 {+0.05/4 1.6} 7. exd5 {-0.05/7 1.0} hxg5 {+0.28/6 1.7} 8. dxc6
{+0.31/8 1.4} Hxc6 {-0.19/6 1.2} 9. Hxg5 {+0.72/7 2.0} Be6 {-0.25/5 1.1}
10. Be2 {+0.76/6 1.4} Qd4 {-0.11/4 1.6} 11. Bf3 {+0.82/7 2.3} e4
{-0.95/4 1.7} 12. Nxe4 {+1.53/6 2.4} Nxe4 {-0.62/4 1.7} 13. Bxe4
{+1.61/6 0.9} Ha5+ {-0.83/4 1.8} 14. c3 {+1.92/6 1.4} Qc5 {-1.82/4 1.5} 15.
b4 {+2.91/7 1.3} Qxc3+ {-1.24/4 1.1} 16. Bd2 {+3.98/8 1.4} Qxa1
{-3.90/5 1.3}

In this position Sjaak II refuses to accept Qxa1.

[Evert]

Will investigate!

[Evert]

Another Sjaak II problem:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.15"]
[Round "10"]
[White "Fairy-Max 4.8V"]
[Black "Sjaak II unknown revision "]
[Result "1-0"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nf3/E {+0.15/6} d6 {-0.16/7 0.8} 2. Nc3/H {+0.00/6 1.1} Nf6/E
{-0.23/7 1.8} 3. Ng5 {-0.19/6 1.2} Nc6/H {+0.08/6 1.8} 4. d3 {+0.07/7 2.2}
e5 {+0.06/6 1.1} 5. Hd2 {+0.10/7 1.2} d5 {+1.15/5 0.9} 6. e4 {-0.16/6 1.9}
h6 {+0.05/4 1.6} 7. exd5 {-0.05/7 1.0} hxg5 {+0.28/6 1.7} 8. dxc6
{+0.31/8 1.4} Hxc6 {-0.19/6 1.2} 9. Hxg5 {+0.72/7 2.0} Be6 {-0.25/5 1.1}
10. Be2 {+0.76/6 1.4} Qd4 {-0.11/4 1.6} 11. Bf3 {+0.82/7 2.3} e4
{-0.95/4 1.7} 12. Nxe4 {+1.53/6 2.4} Nxe4 {-0.62/4 1.7} 13. Bxe4
{+1.61/6 0.9} Ha5+ {-0.83/4 1.8} 14. c3 {+1.92/6 1.4} Qc5 {-1.82/4 1.5} 15.
b4 {+2.91/7 1.3} Qxc3+ {-1.24/4 1.1} 16. Bd2 {+3.98/8 1.4} Qxa1
{-3.90/5 1.3}

In this position Sjaak II refuses to accept Qxa1.

Do you have any way to figure out what version this is exactly? I can't reproduce the issue locally (that is to say, by entering the moves one-by-one). Perhaps this is a side-effect of some of the issues that popped up in the Windows-build thread.

I suppose I could just run a match against Fairy-Max locally and see whether I run into similar problems. How often does the problem show up?

[Evert]

I'll look into these as well, but I suspect the problem is probably related to the bug I metioned here and subsequent posts, which loses some 100-200 elo in self-testing.

I could build a new version from my development tree for testing, but it will have the usual problem of requiring the large C++ runtime DLL.

[Evert]

Another Sjaak II problem:

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2015.01.15"]
[Round "10"]
[White "Fairy-Max 4.8V"]
[Black "Sjaak II unknown revision "]
[Result "1-0"]
[TimeControl "40/60"]
[Variant "seirawan"]

1. Nf3/E {+0.15/6} d6 {-0.16/7 0.8} 2. Nc3/H {+0.00/6 1.1} Nf6/E
{-0.23/7 1.8} 3. Ng5 {-0.19/6 1.2} Nc6/H {+0.08/6 1.8} 4. d3 {+0.07/7 2.2}
e5 {+0.06/6 1.1} 5. Hd2 {+0.10/7 1.2} d5 {+1.15/5 0.9} 6. e4 {-0.16/6 1.9}
h6 {+0.05/4 1.6} 7. exd5 {-0.05/7 1.0} hxg5 {+0.28/6 1.7} 8. dxc6
{+0.31/8 1.4} Hxc6 {-0.19/6 1.2} 9. Hxg5 {+0.72/7 2.0} Be6 {-0.25/5 1.1}
10. Be2 {+0.76/6 1.4} Qd4 {-0.11/4 1.6} 11. Bf3 {+0.82/7 2.3} e4
{-0.95/4 1.7} 12. Nxe4 {+1.53/6 2.4} Nxe4 {-0.62/4 1.7} 13. Bxe4
{+1.61/6 0.9} Ha5+ {-0.83/4 1.8} 14. c3 {+1.92/6 1.4} Qc5 {-1.82/4 1.5} 15.
b4 {+2.91/7 1.3} Qxc3+ {-1.24/4 1.1} 16. Bd2 {+3.98/8 1.4} Qxa1
{-3.90/5 1.3}

In this position Sjaak II refuses to accept Qxa1.

Do you have any way to figure out what version this is exactly? I can't reproduce the issue locally (that is to say, by entering the moves one-by-one). Perhaps this is a side-effect of some of the issues that popped up in the Windows-build thread.

I suppose I could just run a match against Fairy-Max locally and see whether I run into similar problems. How often does the problem show up?

Ok, this is now fixed. There were actually two bugs here, one related to detection of pinned pieces and one caused by the code that generates gating moves (which is the one you found here: it doesn't show up when the position is setup).

There's still some issue with the search though: sometimes the search depth just collapses. I suspect it's running into a fail-high somewhere in the tree and has difficulty resolving it.

[Evert]

There are still some interesting artifacts, so I haven't pushed out a new release yet. Hopefully sometime later today/tonight.

Currently, SjaakII prefers to keep the Hawk and Elephant in holdings for a little while, which makes for an interesting difference in opening play compared to Fairy-Max. However, it also seems to overly penalise the other side for having "poor mobility" of the Hawk and Elephant while its own Hawk and Elephant have 0 mobility but a slightly positive PST.