Crazyhouse tournaments and rating list

[hgm]

I have an interesting method for determining piece values. My usual method of material imbalance game does not work for drop games, as the material turnover is very high there. So if you start one side with better pieces, before you know it these are fighting for the other side.

So I make use of the fact that CrazyWa can handle many piece types, and determies by a table lookup which captured piece corresponds to which hand counter. That way I can define a piece that for white is a Knight, but when captured goeas into the black hand as a Bishop . Which then again, on capture, turns into a white Knight. With such an N-B switching piece, white will have Knights and black will have Bishops for the entire game.

In my preliminary tests I removed the Q-side minors, and gave each side two N-B switchers in hand. The Knights seem to win that significantly.

[hgm]

Amazing! Sjaak sometimes promotes to Rook. I could not imagine it would ever be useful to promote to R or B in a game where stalemate never is an issue, when the upward-compatible Q is available. So CrazyWa only considers promotion to Q and N. And it promotes to N quite a lot! In one game I saw two promotions to N in a row (the second one delivering mate)!

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2016.10.19"]
[Round "5"]
[White "Sjaak II 1.3.1a"]
[Black "CrazyWa 0.0.0"]
[Result "1-0"]
[TimeControl "40/60"]
[Variant "crazyhouse"]

1. d4 {+0.23/9} d5 {+0.00/9 1.1} 2. Nf3 {+0.12/10 1.0} e6 {+0.13/9 1.9} 3.
Nc3 {+0.24/9 1.2} Bb4 {+1.21/9 1.9} 4. Qd3 {+0.30/9 1.5} Bd7 {+1.39/7 0.9}
5. a3 {+0.61/9 1.0} Bxc3+ {+1.37/8 1.4} 6. Qxc3 {+0.73/8 0.9} Ne7
{+1.27/8 3} 7. Bf4 {+0.84/7 0.8} Bc6 {+1.19/7 1.1} 8. B@g3 {+0.56/6 0.9}
N@d6 {+0.25/7 0.9} 9. O-O-O {+0.34/8 1.0} Qd7 {+0.21/8 2.0} 10. e3
{+0.61/8 1.2} Ne4 {+0.36/8 1.0} 11. Qa5 {+0.91/9 1.0} Nxg3 {+0.26/8 1.1}
12. hxg3 {+1.11/8 1.1} b6 {+0.20/8 2.4} 13. Qe1 {+0.82/8 1.1} Qc8
{+0.10/7 1.3} 14. g4 {+1.46/7 1.7} h6 {+0.17/7 2.2} 15. N@b5 {+1.55/7 1.3}
Bxb5 {+1.19/7 1.4} 16. Bxb5+ {+1.42/7 1.3} Kf8 {+0.92/7 1.8} 17. B@e5
{+1.83/8 1.8} a6 {+0.56/7 3} 18. Bd3 {+2.10/7 1.4} B@e4 {+0.31/6 0.8} 19.
Bxc7 {+2.61/7 1.1} Bxf3 {-0.74/7 2.2} 20. gxf3 {+2.90/8 1.4} Nd7
{-1.09/7 2.4} 21. P@g5 {+3.18/6 1.6} Ke8 {+0.10/6 1.1} 22. gxh6
{+3.80/6 1.0} gxh6 {+0.09/6 1.2} 23. P@g7 {+3.78/7 2.0} Rg8 {+0.35/6 1.0}
24. Rxh6 {+3.44/7 2.1} P@a2 {+0.43/5 1.3} 25. B@a1 {+3.36/8 2.5} N@g2
{-0.15/7 2.3} 26. Qb4 {+4.27/8 1.2} a5 {-1.48/6 1.3} 27. Qb5 {+4.77/7 1.0}
N@a7 {-1.73/6 0.7} 28. Qa4 {+10.64/8 1.7} Rxg7 {-3.62/7 2.2} 29. P@h7
{+12.74/8 2.0} P@c4 {-3.77/6 1.8} 30. h8=R+ {+22.59/9 2.0} Rg8
{-7.17/6 0.6} 31. Rxg8+ {+22.05/9 2.0} Nxg8 {-10.94/6 0.6} 32. R@h8
{+22.05/7 1.2} Kf8 {-13.49/6 0.9} 33. Rxg8+ {+22.05/8 2.1} Kxg8
{-6.17/6 0.7} 34. N@e7+ {+21.61/7 1.6} Kf8 {-6.18/6 4} 35. Nxc8
{+22.48/8 2.2} R@b1+ {-7.43/5 1.4} 36. Kd2 {+22.05/2} Rxd1+ {-7.42/4 1.3}
37. Kxd1 {+26.02/6 6} R@e1+ {-7.42/3 0.8} 38. Kd2 {+20.35/2} P@c3+
{-11.67/3 0.4} 39. Kxc3 {+159.93/11 0.9} Nb5+ {-20.12/3 0.3} 40. Qxb5
{+159.95/3} Rxc8 {-79.98/10 0.2} 41. R@g8+ {+159.97/2} Kxg8
{-79.99/101 0.1} 42. Q@h8# {+159.99/2}
{White mates} 1-0

[d]r1q1k3/n1BnnprP/1p2p2R/p2p4/Q1pP1BP1/P2BPP2/pPP2Pn1/B1KR4 w - - 2 30
30. h8=R

[Evert]

This special mate search does not prefer the shortest mate?

That was an interesting puzzle to solve.

The mate search is a normal alpha-beta search, so unless the alpha-beta bounds specify a particular mate depth, any mate score it finds will cause a cut-off (which is fine - the main purpose of the mate search is to detect mate-attacks that refute the intended move but that are missed by the main search). So it's certainly possible that it finds "36. Nf4+ Kh4 37. Rxg4#" before "36. Qxg4#", if move ordering is such that Nf4+ is searched before Qxg4#.
Turns out, that is exactly the case because it considers "Qxg4 Kxg4 Rxg4" a losing sequence of captures, and so it gets sorted low on the move list. The reason for that is that the value of the King is set to 0 rather than something like +CHECKMATE.

So something to fix. I've been thinking of replacing the mate search with something like proof-number search, but I'm not sure it's worth it. Not that proof-number search will necessarily find the fastest mate.

[Evert]

Amazing! Sjaak sometimes promotes to Rook. I could not imagine it would ever be useful to promote to R or B in a game where stalemate never is an issue, when the upward-compatible Q is available.

At least in this example, it's not: the Rook is immediately exchanged, as a Queen would have been.
SjaakII suffers from the problem that it prefers to promote to Rook if it doesn't actually matter because the Rook will be captured anyway. It looks really stupid, but I haven't been able to figure out why it does this. Should be move ordering (if the piece is going to be captured, better to lose a Rook than a Queen), except I'm pretty sure that the value of the promotion piece is not used when calculating the value of the promotion move if it's going to be captured immediately.

I may have another go at fixing this (it's a bit annoying).

[Evert]

In my preliminary tests I removed the Q-side minors, and gave each side two N-B switchers in hand. The Knights seem to win that significantly.

Clever.
I can imagine the Knights being much stronger here, due to the always crowded board.

[hgm]

Indeed, proof-number search finds the mate with the smallest proof tree. But I guess it does not matter so much which mate you find, as long as you stick to it.

I am also considering a small proof-number mate search as part of QS. I have no experience with that, however. But watching games between Sjaak and CrazyWa gives me the idea that the main reason Sjaak beats CrazyWa is that it sees mates much earlier.

[hgm]

At least in this example, it's not: the Rook is immediately exchanged, as a Queen would have been.

But the amazing thing is that it even considers it. In Crazyhouse there should just be two choices: Queen and Knight. Rook or Bishop can at best be as good as Queen, so why waste time to figure out if it actually is?

BTW, I have a game in which Sjaak crashed. Technically it crashed after the game, as WinBoard already declared checkmate. But I am not sure Sjaak agrees with that, its DTM is on the previous move seems a bit high. Unfortunately no debug file was being made. But procedure here is that WinBoard would send 'force' and then the opponents last move, beforesending a 'result' command. It could have chocked on the move.

Code: Select all

[Event "Computer Chess Game"]
[Site "ONTWIKKELLAPTOP"]
[Date "2016.10.19"]
[Round "29"]
[White "Sjaak II 1.3.1a"]
[Black "CrazyWa 0.0.1"]
[Result "0-1"]
[TimeControl "40/60"]
[Variant "crazyhouse"]

1. d4 {+0.22/9} e6 {+0.26/9 1.1} 2. Nc3 {+0.43/9 1.0} Na6 {+0.18/9 1.7} 3.
e4 {+0.69/9 1.1} Nb8 {+0.06/9 1.9} 4. Nf3 {+0.80/8 1.2} Bb4 {+0.22/9 1.5}
5. e5 {+1.01/7 0.9} Kf8 {+0.23/8 1.2} 6. Qd3 {+1.11/7 1.4} Ne7
{+0.32/8 1.5} 7. a3 {+1.17/8 1.0} Ba5 {+0.35/9 4} 8. Be3 {+1.13/7 1.2}
Bxc3+ {+0.57/8 1.7} 9. Qxc3 {+1.09/7 1.6} Nd5 {+0.32/8 2.3} 10. Qb3
{+1.22/7 1.2} h5 {+0.45/7 2.5} 11. O-O-O {+1.07/6 1.3} Nxe3 {+0.52/7 1.3}
12. fxe3 {+1.28/7 1.7} Nc6 {+0.35/6 1.8} 13. d5 {+1.56/6 1.4} exd5
{+0.89/6 1.4} 14. Qxd5 {+1.03/7 1.6} B@b6 {+0.60/5 2.7} 15. B@c5+
{+1.61/5 1.1} Bxc5 {+0.49/6 1.3} 16. Qxc5+ {+1.40/7 1.7} Ke8 {+0.48/5 0.9}
17. N@f5 {+1.48/5 1.7} N@e6 {+0.46/5 1.0} 18. Qd5 {+1.48/6 1.1} B@g4
{+0.65/5 1.5} 19. B@h4 {+1.51/6 1.1} P@g5 {+0.51/6 1.2} 20. P@e7
{+0.05/7 1.4} Nxe7 {+3.35/7 0.8} 21. Nxe7 {-0.27/8 1.0} Qxe7 {+3.39/6 0.9}
22. Be1 {-0.36/7 1.1} Bxf3 {+3.36/5 0.9} 23. Qxf3 {-0.61/7 1.3} P@a2
{+4.08/5 1.1} 24. N@a1 {-0.65/8 2.0} Qc5 {+3.88/6 1.1} 25. Rd5
{-0.59/6 1.0} g4 {+6.56/6 1.0} 26. Qe4 {-0.65/8 2.8} Qxd5 {+6.37/6 0.9} 27.
Qxd5 {-1.72/8 1.0} R@b1+ {+7.82/6 1.9} 28. Kd2 {+3.88/2} Rxa1 {+7.83/5 1.0}
29. B@c1 {-2.83/7 2.7} N@b6 {+11.97/6 2.1} 30. Qe4 {-7.61/7 1.9} Rxc1
{+13.87/6 1.3} 31. Bh4 {-8.00/7 2.4} g5 {+14.98/4 0.9} 32. Qb4
{-10.26/5 1.2} Nd5 {+19.26/5 3} 33. Qc4 {-17.91/5 2.5} a1=Q {+20.74/5 2.6}
34. Bxg5 {-19.87/5 1.1} B@c3+ {+26.47/4 1.7} 35. bxc3 {-159.92/5 1.5} N@e4+
{+79.96/6 0.7} 36. Qxe4 {-159.94/5 0.1} Qxc3+ {+79.97/10 0.7} 37. Kxc1
{-159.96/2} N@b3+ {+79.98/30 0.5} 38. Kd1 {-159.96/2} Qd2# {+79.99/2 0.1}
{Black mates} 0-1

[Evert]

Indeed, proof-number search finds the mate with the smallest proof tree. But I guess it does not matter so much which mate you find, as long as you stick to it.

There's that too.
It enters the mating line in the transposition table, so it should normally pick it up again on the next iteration.

I am also considering a small proof-number mate search as part of QS. I have no experience with that, however.

Nor do I, but it doesn't look too difficult in principle. It's not entirely obvious to me what a good data structure would be though; as a depth-first search you need all parts of the tree that were visited in memory. My first thought was just a list of positions identified by their hash signature, but then the positions need to have their parent stored as well (so you can update the proof/disprove numbers) and that gets tricky when you consider transpositions and repetitions (the latter should be disallowed, I think, or at least care needs to be taken to not store the repeat node as a draw, but just consider it a non-winning child of the parent node).

But watching games between Sjaak and CrazyWa gives me the idea that the main reason Sjaak beats CrazyWa is that it sees mates much earlier.

I'd be surprised if the mate search helps there; it worked well when I tried it on Shogi test positions, but I never measured a big Elo gain from it (there was some though). This was all self-play, which I think is tricky for this sort of thing (both opponents have similar blind spots either way).

That can be tested by disabling the mate search, of course.

I don't know how advanced CrazyWa's evaluation is at the moment; SjaakII has a King Safety term that helps (a bit; I think it's more important in drop variants too).

[Evert]

But the amazing thing is that it even considers it. In Crazyhouse there should just be two choices: Queen and Knight. Rook or Bishop can at best be as good as Queen, so why waste time to figure out if it actually is?

Yeah, well, they're still legal moves, so the move generator generates them. They can be pruned, but it isn't smart enough to know that.
What's weird is that the move ordering does make sure the Queen promotion is searched before the others, so if that isn't good enough, it's a bit mysterious that one of the later move should be good enough - especially if the rook is taken.

BTW, I have a game in which Sjaak crashed. Technically it crashed after the game, as WinBoard already declared checkmate. But I am not sure Sjaak agrees with that, its DTM is on the previous move seems a bit high. Unfortunately no debug file was being made. But procedure here is that WinBoard would send 'force' and then the opponents last move, beforesending a 'result' command. It could have chocked on the move.

Hmm... this one I can't reproduce. It responds to Qd2 with "0-1 {Black mates}", but it does (again) prefer a longer mate than Qd2...

[hgm]

Nor do I, but it doesn't look too difficult in principle. It's not entirely obvious to me what a good data structure would be though; as a depth-first search you need all parts of the tree that were visited in memory. My first thought was just a list of positions identified by their hash signature, but then the positions need to have their parent stored as well (so you can update the proof/disprove numbers) and that gets tricky when you consider transpositions and repetitions (the latter should be disallowed, I think, or at least care needs to be taken to not store the repeat node as a draw, but just consider it a non-winning child of the parent node).

I was thinking of a recursive implementation:

Per node store a list of moves with the proof and disproof number of the node they lead to (initially both 1, if none of the daughters exists yet). Also store the proof and disproof number of the node itself, and the number of the move with the lowest proof number. (I would go for a 'NegaProve' implementation, where proof and disproof get swapped in successive levels, so that the side-to-move is the 'side-to-prove'.) Then recursively walk the branch of all the best moves (only doing MakeMove() of the indicated 'hash move'), and when the recursion unwinds do the back-propagation of the result. (This would sometimes require searching a new best move, if the proof number goes up.)

It is true that this would miss transpositions. But you can check during the recursion if you hit a node that was changed along another branch, by comparing the numbers listed with the moves with the totals listed in the daughters. If they differ, you abort the branch, and first propagate the altered result back to the root, before trying again. As you follow the path of lowest numbers, a transposition must have had a higher number. So if the number goes down, it would go down the same, but still stay worse. If the number goes up, the transposition would look better than it actually is, and might become a 'false best' in the root. But that would be the time you propagate the score back along that line as well.

Repeats are tricky, as they are in normal search.