Crazyhouse status?

[lkaufman]

What is the current status of Crazyhouse chess engines? Which is currently the strongest one can download for a pc, and where to download it? Are any of them using NNUE or neural nets yet? I saw one strong engine on LiChess called "VariantsBot", but I don't know if there is an equivalent version one can download for pc. It is rated a bit lower than the top humans on LiChess, but I don't think that the top humans play rated games with the top engines, so this may be misleading. I imagine that "variantsBot" is much stronger than any human, but I don't know this, does anyone know?

I would expect that a trained NNUE engine would dominate humans even more dramatically than the NNUE engines do in either chess or shogi, since Crazyhouse is so tactical and since humans don't have centuries of theory built up to guide them. But perhaps none exist yet.

Regarding the game itself, my impression, based mostly on looking at engine games, is that the only real drawback to the game is that White almost certainly has a theoretical win and a huge advantage at top engine level, though this is probably not the case at human level. This would limit the top rating for engines playing each other, since they would tend to trade wins with each color swap. Playing with Armageddon rule (Black wins draws) probably wouldn't help much; in the games of "VariantsBot" with at least a one second increment, I saw only a single draw! But between the top engines the White/Black win ratio was huge, probably double-digit. I can think of two remedies for engines vs. engine play:
1. Limit White's choices on his first move (excluding those that give White too big a plus score in top engine play, or perhaps by some simple rule like no initial pawn move to the fourth rank by White.
Or:
2. Adjudicate the game as a win for Black after N moves, whatever value proves fair at top level engine play. Perhaps something like N = 40 or so. This would be analogous to the game of Go, which would be a certain win for the first player at top engine level except for the Komi rule, where a bonus (usually 6.5) is added to the second player's score to equalize the chances. We could call this variant "CrazyGoChess", it would be a hybrid of shogi, Go, and chess!

[Vinvin]

I think it's this one : https://fairy-stockfish.github.io/about/

...especially after the advent of NNUE, and is now powering analysis and play against the computer on several chess (variant) websites, such as lichess.org and pychess.org.

BTW https://www.pychess.org/games is a great site to play !

[Vinvin]

I can think of two remedies for engines vs. engine play:
...

One more remedy is to allow black to play 2 moves in a row in his first move.

[lkaufman]

I can think of two remedies for engines vs. engine play:
...

One more remedy is to allow black to play 2 moves in a row in his first move.

??That just makes Black in effect have the advantage that White usually has; how does that help anything?

Overnight I ran a thousand games of Fairy Stockfish against itself at 2' +1" (single thread) to see the White advantage and draw rate. I was rather surprised, it was 624 wins for White, 328 wins for Black, and 48 draws, a less lopsided result than I expected. But I really doubt that Fairy Stockfish is the strongest engine available for PC, for several reasons: 1.This low win ratio suggests low level play 2. It is apparently NOT using NNUE or any neural net 3. "Stockfish level 8" playing on LiChess loses to "variantsBot" and has lost to at least one human (I suppose it must be using Fairy Stockfish, but I don't know that) 4. I see it play openings like 1.e4 Nc6 2.Nf3 Nf6 3.e5 d5?? 4.Nc3?? and continuing to allow pawn takes knight and not to play it for a couple more moves!! I'm no expert on the game, but obviously knights are worth far more than pawns, especially with no chance of dropping a pawn on the seventh rank any time soon, and based on my expertise in shogi one should play for material advantage until the kings are in danger. This really suggests a weak program (poor eval?).

So what stronger engines are available for the PC?

[MTaktikos]

So what stronger engines are available for the PC?

Almost sure your engine had not access (perhaps wrong path) to the NNUE file.

The stongest engine for Crazyhouse chess is Fairy-Stockfish with crazyhouse-8ebf84784ad2.nnue of Belzedar, which plays 1136 Elo (!) above classic evaluation (https://fairy-stockfish.github.io/nnue/)
The variantBot of lichess is AFAIK nothing else than Fairy-Stockfish (https://lichess.org/forum/lichess-feedb ... -stockfish)

Don't know which user interface you used for the engine match and if it has an ini file where you can edit the path of the nnue file. To make things easier, I made Windows compiles with embedded Crazyhouse net, where the above mentioned net is already predefined, have compiled the engine as avx2 (for AMD CPUs) and bmi2 (for Intel CPUs)
You can download it here:
https://filehorst.de/d/eJnicBzx
If you want to analyse the first-player advantage of the game, of course it is not recommended to run it single-threaded, but give it so many threads as your CPU allows.

The second top engine in strength is CrazyAra, for my GPU 3060Ti it is CrazyAra 0.95 (https://github.com/QueensGambit/CrazyAr ... nsorRT.zip and net https://github.com/QueensGambit/CrazyAr ... -os-96.zip)

[lkaufman]

So what stronger engines are available for the PC?

Almost sure your engine had not access (perhaps wrong path) to the NNUE file.

The stongest engine for Crazyhouse chess is Fairy-Stockfish with crazyhouse-8ebf84784ad2.nnue of Belzedar, which plays 1136 Elo (!) above classic evaluation (https://fairy-stockfish.github.io/nnue/)
The variantBot of lichess is AFAIK nothing else than Fairy-Stockfish (https://lichess.org/forum/lichess-feedb ... -stockfish)

Don't know which user interface you used for the engine match and if it has an ini file where you can edit the path of the nnue file. To make things easier, I made Windows compiles with embedded Crazyhouse net, where the above mentioned net is already predefined, have compiled the engine as avx2 (for AMD CPUs) and bmi2 (for Intel CPUs)
You can download it here:
https://filehorst.de/d/eJnicBzx
If you want to analyse the first-player advantage of the game, of course it is not recommended to run it single-threaded, but give it so many threads as your CPU allows.

The second top engine in strength is CrazyAra, for my GPU 3060Ti it is CrazyAra 0.95 (https://github.com/QueensGambit/CrazyAr ... nsorRT.zip and net https://github.com/QueensGambit/CrazyAr ... -os-96.zip)

Thanks. The embedded net version works properly, and it does crush the no-net version in my own test, but "only" by 363 elo after 100 games at 2' + 1" on four threads (CuteChess gui, four games at once running on 16 thread machine). Nowhere near the claimed 1136 elo. I note that it won every single game with White, but "only" won by about 39 to 11 with Black (no draws!). Perhaps the 1136 elo claim is based on pairs of games with an unequal result, to equalize the color effect, but doing this grossly exaggerates the Elo gain even when there is no color bias. By that criterion I got an infinite elo difference, since NNUE won 39 pairs while no NNUE just split the other 11. Otherwise it is hard to imagine how my result could differ so much from the claimed elo gain, unless it is based on some absurdly fast level of play.

Anyway, even 363 elo is a lot, and confirms what I said about the strength of FSF without a net. Its eval is just really poor I think, but a NN pretty much fixes the problem. I plan to do further experiments on the White advantage, and also on the advantage of 4 vs 2 or 1 threads. I think this is a game where enormous elo gains are likely if someone puts in the effort, especially if Elo is measured by decisive game pairs as I suspect it already is.

[MTaktikos]

after 100 games [...] I note that it won every single game with White

I think we agree that this game must be a first player win

hard to imagine how my result could differ so much from the claimed elo gain, unless it is based on some absurdly fast level of play.

Not sure about Bezedar's test conditions, only that it was indeed bullet time 10'' + 0.1''
In a game with such big or decisive first-player-advantage it shouldn't wonder, that increasing the time, descreases the elo difference nnue vs classical evaluation. If we increase the time, say, to corr chess 3 days/move, then even classical will find the winning lines for White, so the elo difference will decrease to zero.

For further analysis I recommend to install the engine in the GUI liground (https://github.com/ml-research/liground ... .0-win.exe)
With the help of Fairy Stockfish in the GUI liground, I could extract two main lines for White:
1.e4 e5 2.Nf3 Nc6 3.d4 exd4 4.Nxd4 Nf6 5.Nc3 Bb4
and 1.e4 Nf6 2.e5 d5 (or changed move order 2...Ne4 3.d4 d5) 3.d4 Ne4 5.f3 Bf5 6.Bb5+-

that can serve as the first two lines in an opening book for the first-player win

[lkaufman]

after 100 games [...] I note that it won every single game with White

I think we agree that this game must be a first player win

hard to imagine how my result could differ so much from the claimed elo gain, unless it is based on some absurdly fast level of play.

Not sure about Bezedar's test conditions, only that it was indeed bullet time 10'' + 0.1''
In a game with such big or decisive first-player-advantage it shouldn't wonder, that increasing the time, descreases the elo difference nnue vs classical evaluation. If we increase the time, say, to corr chess 3 days/move, then even classical will find the winning lines for White, so the elo difference will decrease to zero.

For further analysis I recommend to install the engine in the GUI liground (https://github.com/ml-research/liground ... .0-win.exe)
With the help of Fairy Stockfish in the GUI liground, I could extract two main lines for White:
1.e4 e5 2.Nf3 Nc6 3.d4 exd4 4.Nxd4 Nf6 5.Nc3 Bb4
and 1.e4 Nf6 2.e5 d5 (or changed move order 2...Ne4 3.d4 d5) 3.d4 Ne4 5.f3 Bf5 6.Bb5+-

that can serve as the first two lines in an opening book for the first-player win

Overnight I ran FSF with the recommended net on selfplay on four threads at 2' + 1", and after 484 games I got 447 wins for White, 24 wins for Black, and 13 draws, an 18.5 to 1 win ratio for White. Clearly some restriction on White's first move or some other rule mod is needed for the game to be considered playable at top engine level.

I downloaded liground; is the "Fairy Stockfish 13" included with it the same as or at least fairly close to the FSF with your recommended net for Crazyhouse? If not, how to I add an engine (there is an "add engine" button, but it's not obvious what to do after clicking it)?

Regarding your above main line, 1.e4 Nf6? 2.e5 d5?? 3.d4??, how can that possibly be good play? I don't know if you are a chess or shogi player, but it is absurd to think that winning a knight for a pawn on move 3 can be bad, even in Crazyhouse. Black blunders the knight, White declines!! I think it must mean that even the NNUE version is trained very heavily on the regular version, which apparently has ridiculous eval. I'm pretty much a novice at Crazyhouse, but I can't believe that even the best human players of the game would play such silly moves. I mean if you can't even assume that in the opening it's good to take a knight for a pawn, how can you even play sensibly? If this were really the right way to play, top humans would have no chance even against a no-net engine doing a very short search. Of course, even with nonsense eval, 25 ply searches will crush any human, but it suggests that a future engine should score 100% against today's best, if we assume that the White advantage is addressed somehow.

Finally, I reran the NNUE vs no NNUE test at the quoted 10" + 0.1" time limit on 4 threads (probably the quoted +1136 elo result was one thread, but as you noted that doesn't produce enough variety to be valid). My result was +978 -20 =2 for +667.4 elo. So as you said faster play minimizes the White advantage and hence exaggerates the elo difference.

[MTaktikos]

I downloaded liground; is the "Fairy Stockfish 13" included with it the same as or at least fairly close to the FSF with your recommended net for Crazyhouse? If not, how to I add an engine (there is an "add engine" button, but it's not obvious what to do after clicking it)?

No, version 13 is not up to date. Here how to install the new engine (because this instructions are also for other readers who want to install an uci-engine in liground, and may even have not found the add-button, I think it's the best to give all the steps explicitly):
1) After starting liground, set the chess variant to "Crazyhouse"
2) Click on "Settings"
3) In the right of the board, you can see the name of one of the installed engines, and in the right of this name, there are icons to edit this engine, to remove this engine from liground, or to add a new engine (+). Click on the add -icon
4) There pops up a mini window. Here you click on "Select path" and, after navigating through your HD folders, double-click on your file FSFCrazyhouse.exe.
5) Then edit in the mini window the first line (Name) with a name of your choice for the Crazyhouse engine (I took FSFCrazyhouse). And then click on the green button "Save" in the mini window
6) Then you set the mouse on the little arrow directly right of the engine name, and you can chose the name of the new defined engine (in my case, FSFCrazyhouse)
7) Continue to edit the settings, on my PC I edited Threads to 24 (instead of 1) and Hash to 4096 (instead of 16). Usually there is to edit also the nnue path, but in the engine I compiled it's predefined. Then click on the green "Save" button
8) Now the engine starts, you see below some extremely small lines that give the cmd output. Wait some seconds until no new lines are printed, then your engine is ready to analyze
9) Then you go to the big button in the right of the engine name (FSFCrazyhouse), click on it, the analysis starts, and you can move the pieces on the board and also take moves back, while the analysis is adapted to every new position

Regarding your above main line, 1.e4 Nf6? 2.e5 d5?? 3.d4??, how can that possibly be good play? I don't know if you are a chess or shogi player, but it is absurd to think that winning a knight for a pawn on move 3 can be bad, even in Crazyhouse. Black blunders the knight, White declines!!

Tried to move back and forth, but FSF remains in this line and evaluates White's advantage as +1.94, see the picture below


OK, let's not decline the knight capture, we move 3. e5xf6. The liground analysis gives FSF's answer 3... e7xf6, and then, looking at the evaluation, hm, FSF obviously "thinks" that White blundered and that Black can now reach a draw. Are you sure, that this must be an evaluation error, or is it possible, that FSF can see something that we don't see?

[lkaufman]

I downloaded liground; is the "Fairy Stockfish 13" included with it the same as or at least fairly close to the FSF with your recommended net for Crazyhouse? If not, how to I add an engine (there is an "add engine" button, but it's not obvious what to do after clicking it)?

No, version 13 is not up to date. Here how to install the new engine (because this instructions are also for other readers who want to install an uci-engine in liground, and may even have not found the add-button, I think it's the best to give all the steps explicitly):
1) After starting liground, set the chess variant to "Crazyhouse"
2) Click on "Settings"
3) In the right of the board, you can see the name of one of the installed engines, and in the right of this name, there are icons to edit this engine, to remove this engine from liground, or to add a new engine (+). Click on the add -icon
4) There pops up a mini window. Here you click on "Select path" and, after navigating through your HD folders, double-click on your file FSFCrazyhouse.exe.
5) Then edit in the mini window the first line (Name) with a name of your choice for the Crazyhouse engine (I took FSFCrazyhouse). And then click on the green button "Save" in the mini window
6) Then you set the mouse on the little arrow directly right of the engine name, and you can chose the name of the new defined engine (in my case, FSFCrazyhouse)
7) Continue to edit the settings, on my PC I edited Threads to 24 (instead of 1) and Hash to 4096 (instead of 16). Usually there is to edit also the nnue path, but in the engine I compiled it's predefined. Then click on the green "Save" button
8) Now the engine starts, you see below some extremely small lines that give the cmd output. Wait some seconds until no new lines are printed, then your engine is ready to analyze
9) Then you go to the big button in the right of the engine name (FSFCrazyhouse), click on it, the analysis starts, and you can move the pieces on the board and also take moves back, while the analysis is adapted to every new position

Regarding your above main line, 1.e4 Nf6? 2.e5 d5?? 3.d4??, how can that possibly be good play? I don't know if you are a chess or shogi player, but it is absurd to think that winning a knight for a pawn on move 3 can be bad, even in Crazyhouse. Black blunders the knight, White declines!!

Tried to move back and forth, but FSF remains in this line and evaluates White's advantage as +1.94, see the picture below


OK, let's not decline the knight capture, we move 3. e5xf6. The liground analysis gives FSF's answer 3... e7xf6, and then, looking at the evaluation, hm, FSF obviously "thinks" that White blundered and that Black can now reach a draw. Are you sure, that this must be an evaluation error, or is it possible, that FSF can see something that we don't see?

Thanks, got it working per your instructions. Regarding the above 1.e4 Nf6 2.e5 d5 3.d4, now that I've studied the lines and learned a bit more about the game, I'm inclined to think that it's not so much a question of specific lines that it sees, but rather the incredible value of tempi and mobility and king safety. If White plays the obvious 3.exf6 and Black plays e7xf6, Black has gained two tempi (...d5 and e7 to f6; the pawn is far better on f6 than e7, not blocking the bishop but guarding critical squares e5 and g5). So basically Black got a pawn and two moves for the knight. In chess that's not nearly enough, but it seems that in Crazyhouse a tempo early in the game is worth something like a pawn, so it's sort of like three pawns for a knight, which might be enough in Crazyhouse, I don't know. As a shogi master this seems bizarre to me, but in shogi pawns in hand are far less valuable than in chess due to the "no doubled pawns allowed" shogi rule. While I'm still very skeptical that this line of play (perhaps hard to avoid after 1...Nf6) is actually a good defense by Black, I must admit that I could well be wrong; perhaps since other first moves also lose, 1...Nf6 isn't clearly worse, and perhaps 2...d5 is better than moving the knight again. Still it seems bizarre not to take it!

Regarding the White advantage, it seems that the four most natural first moves (1.e4, 1.d4, 1.Nf3, and 1.Nc3) all retain a probably winning advantage, while if you trust FSF NNUE the other 16 moves all favor Black at least slightly. 1.e3 seems to be the closest to equal, though perhaps not close enough to be called "balanced". So it seems that there is no restriction on White's first move that really makes the game equal based on this engine. Of course longer sequences are very equal, for example 1.e4 e5 2.d3 seems to be a virtual toss-up, but that's not a very elegant way to start a game. Simply forbidding White from short castling also shows very nearly a zero score, but again not very elegant. A "Komi" rule like "Black wins after 40 moves are completed" (or whatever proves fair) might be the best way to play balanced Crazyhouse. Perhaps there are other ideas. It seems to me that a balanced offshoot of Crazyhouse would be a very interesting game, even for engine competition.