List of bugfree, opensource Linux and MacOSX engines

[OliverBr]

This should be cured in version 5, which invalidates its hash table before any search where the use of null move was switched.

The latest source of Fairy-Max can be found in my git repository:

http://hgm.nubati.net/cgi-bin/gitweb.cgi

Just download the latest snapshot of the main branch. Same for KingSlayer / simple.

I have another game with Fairy-Max 5.0b, when it is stalling. It is quite a large game, perhaps it can help you:

[pgn]
[Event "?"]
[Site "?"]
[Date "2020.11.11"]
[Round "2296"]
[White "Fairy-Max 5.0b"]
[Black "OliThink 4.1.3"]
[Result "0-1"]
[ECO "A13"]
[GameDuration "00:01:18"]
[GameEndTime "2020-11-11T12:16:22.495 CET"]
[GameStartTime "2020-11-11T12:15:03.676 CET"]
[Opening "English Opening"]
[PlyCount "218"]
[Termination "stalled connection"]
[TimeControl "40/12"]

1. c4 {book} e6 {book} 2. Nc3 {book} c6 {book} 3. b3 {book} d5 {book}
4. f4 {-0.04/6 0.26s} dxc4 {+0.46/8 0.28s} 5. bxc4 {-0.23/6 0.24s}
Bd6 {+0.44/8 0.28s} 6. Nh3 {-0.22/6 0.19s} e5 {+1.14/9 0.23s}
7. fxe5 {-0.23/7 0.35s} Bxe5 {+0.89/8 0.18s} 8. Qb3 {-0.24/6 0.27s}
Nd7 {+1.13/7 0.27s} 9. Nf2 {-0.02/5 0.21s} Nc5 {+1.12/8 0.42s}
10. Qb4 {-0.06/6 0.24s} Bd6 {+0.91/8 0.45s} 11. Qb1 {-0.09/6 0.20s}
Nf6 {+1.15/7 0.25s} 12. d4 {-0.15/6 0.17s} Ne6 {+1.07/7 0.22s}
13. Be3 {-0.06/7 0.99s} Qa5 {+1.15/6 0.17s} 14. Qb3 {-0.20/6 0.26s}
c5 {+1.15/6 0.17s} 15. Qb5+ {+0.45/9 0.38s} Qxb5 {+0.73/9 0.23s}
16. Nxb5 {+0.66/8 0.22s} Ke7 {+0.52/9 0.28s} 17. Nxd6 {+1.10/8 0.17s}
Kxd6 {+0.45/9 0.24s} 18. d5 {+0.90/8 0.39s} Nd4 {+0.50/8 0.25s}
19. O-O-O {+0.97/7 0.17s} Ke7 {+0.47/7 0.18s} 20. Bxd4 {+0.82/7 0.22s}
cxd4 {-0.23/10 0.32s} 21. Rxd4 {+0.92/8 0.20s} h5 {-0.31/9 0.32s}
22. d6+ {+1.59/7 0.18s} Ke6 {-0.40/8 0.31s} 23. Nh3 {+1.52/7 0.22s}
Ke5 {+0.39/9 0.31s} 24. e3 {+1.51/7 0.24s} Bxh3 {+0.20/9 0.30s}
25. gxh3 {+1.65/8 1.0s} Ne4 {+0.05/8 0.21s} 26. d7 {+2.34/7 0.17s}
Nd6 {-0.02/7 0.16s} 27. Bg2 {+2.63/7 0.15s} Nf5 {-1.02/7 0.15s}
28. Re4+ {+2.73/7 0.16s} Kf6 {-1.25/7 0.17s} 29. Rf4 {+3.10/8 0.15s}
g5 {-1.49/8 0.19s} 30. Rf2 {+3.12/9 0.67s} Ke6 {-1.69/7 0.16s}
31. Bd5+ {+2.96/8 0.69s} Ke5 {-1.11/6 0.098s} 32. Rhf1 {+2.69/8 0.51s}
Nd6 {-0.90/7 0.19s} 33. Bxf7 {+2.71/8 0.38s} Rh7 {-0.59/7 0.12s}
34. Rf6 {+2.81/8 0.12s} Nxf7 {-0.57/8 0.13s} 35. R1f5+ {+3.12/9 0.13s} Ke4
36. Rxf7 {+2.69/10 0.36s} Rxf7 {-0.51/8 0.15s} 37. Rxf7 {+2.76/10 0.14s}
Rd8 {-0.52/8 0.20s} 38. Re7+ {+2.24/10 0.31s} Kf5 {-0.53/7 0.12s}
39. e4+ {+0.91/9 0.10s} Kf6 {0.00/8 0.11s} 40. Re8 {+0.86/11 0.10s}
Rxd7 {+0.50/8 0.13s} 41. e5+ {+0.86/10 0.19s} Kf5 {+0.81/8 0.42s}
42. e6 {+0.23/10 0.54s} Rd4 {+0.51/9 0.42s} 43. e7 {+0.78/10 0.26s}
Rxc4+ {+0.51/8 0.31s} 44. Kd2 {+0.83/10 0.36s} Kf6 {+0.45/8 0.24s}
45. Rb8 {+0.72/11 0.19s} Kxe7 {+0.41/8 0.57s} 46. Rxb7+ {+0.69/10 0.29s}
Kf6 {+0.46/8 0.37s} 47. Rxa7 {+0.74/10 0.45s} Rd4+ {+0.42/8 0.30s}
48. Ke2 {+0.73/10 0.31s} Rb4 {+0.41/8 0.21s} 49. Ra6+ {+0.68/10 0.27s}
Kf5 {+0.30/8 0.23s} 50. Ra5+ {+0.66/10 0.35s} Kf4 {+0.20/9 0.51s}
51. Kf1 {+0.65/10 0.21s} Rb2 {+0.39/9 0.41s} 52. Kg1 {+0.65/11 0.24s}
Rc2 {+0.33/9 0.39s} 53. Ra4+ {+0.75/11 0.27s} Ke3 {+0.28/8 0.17s}
54. h4 {+0.77/11 0.23s} g4 {+0.11/9 0.30s} 55. Ra7 {+0.74/11 0.32s}
Kd4 {+0.15/8 0.16s} 56. a4 {+0.86/10 0.25s} Ra2 {+0.13/8 0.16s}
57. a5 {+0.80/11 0.49s} Ra3 {+0.10/8 0.20s} 58. Kf2 {+0.87/10 0.26s}
Rh3 {+0.20/8 0.17s} 59. Kg2 {+0.79/11 0.20s} Rxh4 {+0.22/9 0.31s}
60. a6 {+1.21/10 0.17s} Rh3 {+0.22/9 0.33s} 61. Rh7 {+0.78/10 0.20s}
Ke4 {+0.28/9 0.45s} 62. a7 {+0.80/10 0.21s} Ra3 {+0.28/7 0.19s}
63. Kf2 {+0.80/10 0.23s} Kf4 {+0.22/7 0.12s} 64. Rf7+ {+0.77/9 0.17s}
Ke5 {+0.28/7 0.23s} 65. Rd7 {+0.70/9 0.23s} h4 {+0.28/7 0.13s}
66. Re7+ {+0.69/11 0.35s} Kf6 {+0.26/7 0.28s} 67. Rb7 {+0.65/11 0.64s}
Ke6 {+0.26/7 0.20s} 68. Rh7 {+0.66/10 0.17s} Kf5 {+0.26/7 0.15s}
69. Rf7+ {+0.67/11 0.21s} Ke4 {+0.30/7 0.12s} 70. Re7+ {+0.69/11 0.17s}
Kd5 {+0.30/7 0.30s} 71. Rg7 {+0.66/10 0.42s} h3 {+0.30/8 0.31s}
72. Rc7 {-0.45/11 0.33s} Kd6 {+0.36/8 0.34s} 73. Rg7 {-0.50/11 0.16s}
Kc6 {+0.37/8 0.32s} 74. Rf7 {+0.03/11 0.26s} Kb6 {+0.45/8 0.30s}
75. Rh7 {+0.02/11 0.27s} Kc5 {+0.26/8 0.27s} 76. Re7 {+0.04/11 0.23s}
Kd4 {+0.26/7 0.17s} 77. Rc7 {+0.63/12 0.46s} Kd5 {+0.26/7 0.13s}
78. Ke2 {+0.61/11 0.16s} Kd6 {+0.26/7 0.090s} 79. Rf7 {+0.07/11 0.22s}
Ke6 {+0.26/8 0.20s} 80. Rg7 {+0.08/11 0.16s} Kf6 {+0.26/8 0.20s}
81. Rh7 {+0.61/11 0.32s} Ke5 {+0.20/8 0.38s} 82. Kd2 {+0.08/11 0.41s}
Kd4 {+0.28/8 0.58s} 83. Rg7 {+0.07/11 0.35s} Ra2+ {+0.43/8 0.34s}
84. Kc1 {+0.04/12 0.31s} Kc5 {+0.35/8 0.37s} 85. Kd1 {+0.03/11 0.21s}
Kb6 {+0.80/8 0.32s} 86. Ke1 {+0.07/12 0.29s} Rxa7 {+0.55/8 0.28s}
87. Rxa7 {+0.73/14 0.29s} Kxa7 {-0.66/13 0.29s} 88. Kf2 {+1.24/20 0.19s}
Kb8 {-1.08/17 0.47s} 89. Kg3 {+1.33/20 0.24s} Kc7 {-1.23/15 0.21s}
90. Kxg4 {+1.44/21 0.18s} Kd8 {-1.29/15 0.47s} 91. Kxh3 {+1.39/24 0.29s}
Ke7 {-1.49/16 0.51s} 92. Kg4 {+0.91/18 0.19s} Kf8 {-1.48/15 0.25s}
93. h4 {+0.76/17 0.24s} Kg8 {-1.64/16 0.28s} 94. Kf4 {+1.28/17 0.20s}
Kf7 {-1.64/16 0.27s} 95. Ke3 {+0.32/17 0.19s} Kg7 {-0.86/13 0.30s}
96. Kf3 {+0.18/18 0.39s} Kg6 {0.00/13 0.23s} 97. Kf2 {+0.18/17 0.59s}
Kh7 {-0.66/14 0.21s} 98. Kg3 {+1.27/18 0.23s} Kg6 {-1.64/17 0.20s}
99. Kg4 {+0.23/16 0.35s} Kh7 {-1.64/17 0.21s} 100. Kh5 {+0.20/19 1.2s}
Kg7 {-1.64/19 0.27s} 101. Kg5 {+0.52/20 0.20s} Kh8 {-1.64/18 0.33s}
102. Kh6 {+1.12/19 0.18s} Kg8 103. Kg6 {+0.26/18 0.18s} Kf8 {-1.64/17 0.37s}
104. h5 {+0.52/18 0.18s} Kg8 {-1.64/17 0.32s} 105. Kf6 {+0.40/19 0.19s}
Kh8 {-1.64/18 0.32s} 106. Kf5 {+0.11/19 0.22s} Kg8 {-1.64/18 0.35s}
107. Kf4 {+0.51/16 0.20s} Kf8 {-1.64/16 0.33s} 108. Kf3 {+0.09/15 0.34s}
Ke8 {-1.64/15 0.13s} 109. Kf2 {+0.07/15 0.19s}
Kf7 {-1.64/14 0.19s, White's connection stalls} 0-1
[/pgn]

[OliverBr]

I also want to make it on your list.

With latest commit Rubi should compile and run in MacOS. I don't have a Mac but I could compile and run it in a VM.
If you still have problems with latest master, please report here: https://github.com/Matthies/RubiChess/issues/248

Regards, Andreas

I have just pulled the newest version and it compiles smoothly and will be added to the list:

Code: Select all

make
rm -f RubiChess-BMI2 RubiChess-AVX2 RubiChess RubiChess-Legacy *.o
rm -f -rf OPT
rm -f *.clangprof-raw *.profdata
  Compiling RubiChess...
g++ -std=c++11 -Wall -O3 -flto -pedantic -Wextra -Wshadow  -mssse3 -msse2 -mmmx -mpopcnt *.cpp -pthread  -D GITVER=\"20201111084233\" -D GITID=\"99e2c8f\" -DUSE_SSSE3 -DUSE_SSE2 -DUSE_MMX -DUSE_POPCNT -o RubiChess

EDIT: It's no issue, maybe a hint: the name is quite large:

Code: Select all

Score of RubiChess 1.9-dev 20201111084233 (popcnt) vs OliThink 5.9.0: 1291 - 22 - 101  [0.949] 1414

[OliverBr]

I am afraid that cutechess-cli is often the one at fault in your tests, not the engines you analysed.

No, this is not correct.
I let Cutechess-cli play many millions (literally!) games with OlIThink and other engines like Stockfish, Glaurung, Fruit, Weiss (since 1.0) and others and they did not crash even once.
So, no, cutechess-cli is not faulty in the sense that engines fail the test because of it.

[OliverBr]

I presume you are excluding Stockfish derivatives, though you did not put that into your criteria. Otherwise you could include Cfish and corchess (at least). Also lc0 of course.

This is true, too and I forget to mention it. Thank you.
My tests run on a Linux 32-core server without any GPU and Leela can't play her full potential there, so I had to to skip GPU engines, too. Unfortunately.

[mvanthoor]

Well... to some extend, I know how you feel with regard to other engines. I've been trying to find engines in the 1500 to 1900 range to create my first testing pool for Rustic.

I've tried many (after having to actually find them in a very round-about way, because all the download links are broken), and most of them are severely lacking. It often seems they've just been developed so the author could have his own engine a rating list, and then abandoned.

They either:
- crash during games or stop responding
- have massive bugs in their uci/xboard output (or no output at all except for their best move)
- dont comply to specs (not even reacting to "uci" for example)
- have glaring bugs (claiming to have a TT but no indication of using it because search depth is lower than Rustic, who doesn't have a TT yet, and no options to set it, repetition detection bugs, evaluation bugs that make it do strange moves)

Lots of junk in the 1500 - 1900 category, I'm sorry to say; but I have found a few good ones. Against some Rustic greatly overperforms (for example, against Pulse 1.7.2 it can draw games it should have lost because Pulse has a bug in its repetition detection), and against some it underperforms (such as against TSCP, where Rustic loses because it has no evaluation yet, but also not a search depth advantage.)

I'll have to keep trying engines until I find about 10-15 that are stable enough to run some tournaments against, to establish my baseline; before I can start upgrading to the better, more developed engines in 1800-2300 and later 2000-2500 categories.

[OliverBr]

against Pulse 1.7.2 it can draw games it should have lost because Pulse has a bug in its repetition detection)

Oh, I like Pulse. Pulse 2.0.0 ist quite nice, but has a bug that it crashes, when any player has exactly time == 0.


See here: I hope it will be repaired soon: https://github.com/fluxroot/pulse/issues/84

[hgm]

Thanks for the Fairy-Max games. I try to reproduce the error now, but so far without success. What hash-table size are you using?

What does CuteChess mean anyway when it says 'connection stalled'? Does it mean the engine process terminated unexpectedly, or is it just a time forfeit?

I sometimes do see pathologic behavior w.r.t. time usage, but I have never seen a crash. E.g. for move 109 of the game above (a KPK position) starting with an empty hash table, it finishes the PV at depth 22 in 0.64 sec, but then takes 45 sec to get that PV at 23 ply. (And needs 4 min to finish the 23-ply iteration, after changin the PV there after 1:36.)

Code: Select all

 23	+0.28 	772.3M	4:00.25	109. Kg4 Kf8 110. h6 Kg8 111. Kg5 Kh7 112. Kh5 Kh8 
 23	+0.13 	306.0M	1:36.23	109. Kf2 Kf7 110. Kg3 Ke7 
 23	+0.12 	149.2M	0:45.33	109. Kg3 Ke7 110. Kh3 Kf6 111. Kg4 Ke6 
 22	+0.27 	1.89M  	0:00.64	109. Kg3 Ke7 110. Kg4 Kf7 111. Kg3 
 22	+0.26 	1.88M  	0:00.64	109. Kf2 Kf7 110. Kg3 Ke7 
 21	+0.26 	1.47M  	0:00.51	109. Kf2 Kf7 110. Kg3 Ke7 
 21	+0.25 	1.46M  	0:00.50	109. Kg3 Ke7 110. Kg4 Kf7 111. Kg5 Ke7 112. h6 Kf7 
 20	+0.20 	1.07M  	0:00.37	109. Kg3 Ke7 110. Kh3 Ke6 111. Kg4 Kf7 112. Kg3 
 20	+0.18 	1.06M  	0:00.37	109. Kf2 Kf7 110. Kg3 Ke7 

So it can apparently happen that there is an extreme factor between the time usage in one iteration and the next, while on all previous iterations the EBF was only 1.3 or so. This makes the "always finish the iteration" time management a bust; you cannot possibly prepare for that. But it would cause forfeits, not real crashes.

I guess I really have to abort thinking, through some 'cold-turkey timeout' to make this problem go away.

[Ras]

What does CuteChess mean anyway when it says 'connection stalled'? Does it mean the engine process terminated unexpectedly, or is it just a time forfeit?

From https://github.com/cutechess/cutechess/ ... sult.h#L56:

Code: Select all

//! Loser's connection stalls (doesn't respond to ping).
StalledConnection,

That might even be related to the time problem if the engine forfeits and Cutechess tries to ping it, to which the engine doesn't answer because it doesn't evaluate input during search.

[Guenther]

Thanks for the Fairy-Max games. I try to reproduce the error now, but so far without success. What hash-table size are you using?

What does CuteChess mean anyway when it says 'connection stalled'? Does it mean the engine process terminated unexpectedly, or is it just a time forfeit?

Generally, this either means, it timed out before the first move, or it did not respond anymore during a game (probably crashed,
or unresponsive). Time forfeit itself would be a different termination tag for cutechess.

BTW I am a bit confused about the version number, because the latest compilation you offered yourself calls itself 5.0b3,
while the repo seems still to say 5.0b. No idea how I should call it, if I compile it myself?

your compilation 2016-09

Code: Select all

tellics say     Fairy-Max 5.0b3
tellics say     by H.G. Muller

Edit: In case you forgot about it - it is from here ;-)
http://hgm.nubati.net/Fairy-Max.zip

[jdart]

Arasan is very stable and bugfree, but unfortunately I am having problems to compile it in MacOSX and clang.

"make CC=clang" should work.