List of bugfree, opensource Linux and MacOSX engines

[OliverBr]

Hello
I checked a lot of open source engines, if they met the following criterias:

1) Open source (most are on git anyway)
2) Easy compile with make/cmake on both, Linux and MacOSX, with g++ and clang, telling their name on init.
3) Complete implementation of either UCI or CECP protocol
4) Stable over thousands of games
5) Bugfree in terms of timeouts and illegal moves and draw repitions.

From over 1000 engines I found, only 48 met all the criterias.
This is the list, the third column is the relative strength when playing h2h against OliThink 5.9.0 on very fast time control:

Code: Select all

cinnamon	uci	-690	2.3.3	https://github.com/gekomad/Cinnamon
chesley		xboard	-800	1.2.x	https://github.com/matthewgingell/chesley
claudia		uci	-1000	0.5.1	https://github.com/antoniogarro/Claudia
ct800 		uci 	-480	0.41	https://www.ct800.net/download.htm			(make..sh)
defenchess	uci	+600	2.3	https://github.com/cetincan0/Defenchess
demolito	uci	+450	.	https://github.com/lucasart/Demolito
delocto		uci	+10	0.6	https://github.com/moterink/Delocto
drosophila	uci	-210	1.6	https://github.com/gustafullberg/drosophila		(cmake)
ethereal 	uci	+700	12.55	https://github.com/AndyGrant/Ethereal
floyd		uci	-750	0.9	https://github.com/kervinck/floyd
fridolin3	uci	-200	3.1	https://sites.google.com/site/fridolinchess
fruit		uci	-180	2.1	https://github.com/Warpten/Fruit-2.1
glaurung	uci	-40	2.2	https://github.com/phenri/glaurung
googleplex	uci	-500	1.7	https://gitlab.com/tsoj/Googleplex_Starthinker		(cmake)
greko 		xb+uci	-150	2020.03	http://greko.su/index_en.html
greko-junior	xb+uci	-490	12.9	https://github.com/coderxlsn/greko-junior-chess
gully2		xboard	-860	2.16	https://github.com/borriss/gully
halogen		uci	-15	8	https://github.com/KierenP/Halogen
k2 		xb+uci	-80	0.96	https://github.com/serg-meus/k2
laser		uci	+500	1.8	https://github.com/jeffreyan11/laser-chess-engine
magic 		xb+uci 	-670	2	https://github.com/zzhangb4/MagiC-engine-chess
marvin		xb+uci	-220	5.0.0	https://github.com/bmdanielsson/marvin-chess
minnow		uci	-1250	150301	https://github.com/tm512/minnow
monolith	uci	+220	2	https://github.com/cimarronOST/Monolith
olithink5	xboard		5.9.0	https://github.com/olithink/OliThink		(/oliThink4 4.1.3: -700, /oliThink3 3.0.7: -950)
phalanx		xboard 	-180	XXV	https://github.com/student-t/Phalanx-XXIII
prophet4	xboard	-950	4	https://github.com/jswaff/prophet4
protector	xb+uci	+280	1.9.0	https://github.com/MichaelB7/Protector
purplehaze	xboard	-920	2.1.0	https://github.com/vinc/purplehaze
robocide	uci	-550	.	https://github.com/DanielWhite94/robocide
rodentII	uci	+80	0.9.68	https://github.com/nescitus/Rodent_II
sayuri		uci	-1000	18.5.23	https://github.com/MetalPhaeton/sayuri			(cmake)
sbchess		uci	-1100	.	https://github.com/DoubleEspresso/chess
scorpio 	xb+uci	-130	3.0.9	https://github.com/dshawul/Scorpio
senpai		uci	+40	2.0	https://github.com/B4dT0bi/senpai
snowy		uci	-850	0.2.5	https://github.com/JasonCreighton/snowy
stash		uci	-160	21.1	https://gitlab.com/mhouppin/stash-bot
stockfish 	uci	+920	11	https://github.com/official-stockfish/Stockfish
sungorus	uci	-510	1.4	https://sites.google.com/site/sungorus
taltos		xb+uci	-440	.	https://github.com/GBuella/Taltos
teki		uci	-600	.	https://github.com/Mk-Chan/Teki				(cmake)
texel		uci	+330	1.08	https://github.com/peterosterlund2/texel		(cmake)
togaII 		uci	-70	1.2.1a	https://github.com/zwegner/toga-mp
weiss 		uci	+250	1.1	https://github.com/TerjeKir/weiss
winter 		uci 	+300	0.8.16	https://github.com/rosenthj/Winter
wyldchess 	uci	-160	1.5	https://github.com/Mk-Chan/WyldChess
xiphos 		uci	+620	0.6.1	https://github.com/milostatarevic/xiphos
zevra2		uci	-370	2.1.2	https://github.com/sovaz1997/Zevra2		(/Zevra 1.8.6: -550)

[OliverBr]

For completition I repost the list ordered by strength:

Code: Select all

stockfish       uci     +920    11      https://github.com/official-stockfish/Stockfish
ethereal        uci     +700    12.55   https://github.com/AndyGrant/Ethereal
xiphos          uci     +620    0.6.1   https://github.com/milostatarevic/xiphos
defenchess      uci     +600    2.3     https://github.com/cetincan0/Defenchess
laser           uci     +500    1.8     https://github.com/jeffreyan11/laser-chess-engine
demolito        uci     +450    .       https://github.com/lucasart/Demolito
texel           uci     +330    1.08    https://github.com/peterosterlund2/texel                (cmake)
winter          uci     +300    0.8.16  https://github.com/rosenthj/Winter
protector       xb+uci  +280    1.9.0   https://github.com/MichaelB7/Protector
weiss           uci     +250    1.1     https://github.com/TerjeKir/weiss
monolith        uci     +220    2       https://github.com/cimarronOST/Monolith
rodentII        uci     +80     0.9.68  https://github.com/nescitus/Rodent_II
senpai          uci     +40     2.0     https://github.com/B4dT0bi/senpai
delocto         uci     +10     0.6     https://github.com/moterink/Delocto
olithink5       xboard          5.9.0   https://github.com/olithink/OliThink            (/oliThink4 4.1.3: -700, /oliThink3 3.0.7: -950)
halogen         uci     -15     8       https://github.com/KierenP/Halogen
glaurung        uci     -40     2.2     https://github.com/phenri/glaurung
togaII          uci     -70     1.2.1a  https://github.com/zwegner/toga-mp
k2              xb+uci  -80     0.96    https://github.com/serg-meus/k2
scorpio         xb+uci  -130    3.0.9   https://github.com/dshawul/Scorpio
greko           xb+uci  -150    2020.03 http://greko.su/index_en.html
stash           uci     -160    21.1    https://gitlab.com/mhouppin/stash-bot
wyldchess       uci     -160    1.5     https://github.com/Mk-Chan/WyldChess
fruit           uci     -180    2.1     https://github.com/Warpten/Fruit-2.1
phalanx         xboard  -180    XXV     https://github.com/student-t/Phalanx-XXIII
fridolin3       uci     -200    3.1     https://sites.google.com/site/fridolinchess
drosophila      uci     -210    1.6     https://github.com/gustafullberg/drosophila             (cmake)
marvin          xb+uci  -220    5.0.0   https://github.com/bmdanielsson/marvin-chess
zevra2          uci     -370    2.1.2   https://github.com/sovaz1997/Zevra2             (/Zevra 1.8.6: -550)
taltos          xb+uci  -440    .       https://github.com/GBuella/Taltos
ct800           uci     -480    0.41    https://www.ct800.net/download.htm                      (make..sh)
greko-junior    xb+uci  -490    12.9    https://github.com/coderxlsn/greko-junior-chess
googleplex      uci     -500    1.7     https://gitlab.com/tsoj/Googleplex_Starthinker          (cmake)
sungorus        uci     -510    1.4     https://sites.google.com/site/sungorus
robocide        uci     -550    .       https://github.com/DanielWhite94/robocide
teki            uci     -600    .       https://github.com/Mk-Chan/Teki                         (cmake)
magic           xb+uci  -670    2       https://github.com/zzhangb4/MagiC-engine-chess
cinnamon        uci     -690    2.3.3   https://github.com/gekomad/Cinnamon
floyd           uci     -750    0.9     https://github.com/kervinck/floyd
chesley         xboard  -800    1.2.x   https://github.com/matthewgingell/chesley
snowy           uci     -850    0.2.5   https://github.com/JasonCreighton/snowy
gully2          xboard  -860    2.16    https://github.com/borriss/gully
purplehaze      xboard  -920    2.1.0   https://github.com/vinc/purplehaze
prophet4        xboard  -950    4       https://github.com/jswaff/prophet4
claudia         uci     -1000   0.5.1   https://github.com/antoniogarro/Claudia
sayuri          uci     -1000   18.5.23 https://github.com/MetalPhaeton/sayuri                  (cmake)
sbchess         uci     -1100   .       https://github.com/DoubleEspresso/chess
minnow          uci     -1250   150301  https://github.com/tm512/minnow

And there is a list of 26 candidates, quite interesting engines, which do compile on both Linux and MacOSX and are stable, but have some flaws, which should be repaired easily. Very prominent is tscp, which only lack telling its name:

Code: Select all

achillees       uci*    -800    .       https://github.com/Tempate/Achillees                    (no name)
bullitchess     uci     -1200   1.0.1   https://bitbucket.org/CornFLX/bullitchess/src/master    (cmake, time: 0,2s/move)
clevergirl      uci     -700    .       https://github.com/SethKasmann/clever-girl              (time: 0,5s/move)
darktemplar     xboard* -2000   ?       https://github.com/bineshandrews/darktemplar-chess      (no name, time: ~0,5s/move)
diablo          uci     -440    0.5.1   https://github.com/mprewarski/DiabloChessEngine         (Makefile: remove -D_WINDOWS_)
hippocampe      xboard* -2150   0.4.2   http://www.computer-chess.org/lib/exe/fetch.php?media=computer_chess:wiki:download:hippocampe-0.4.2.tar.gz (no name, ill.moves)
mchess2         uci     -1800   1.02    https://github.com/MJoergen/mchess2                     (moves instantly, many pv errors)
minimardi       xboard* -1000   1.3     https://sourceforge.net/projects/minimardi              (no name)
mscp            xboard* -1200   1.4     https://marcelk.net/mscp/download                       (no name, moves instantly)
napoleon        uci     -800    1.7.0   https://github.com/crybot/Napoleon                      (boost framework to compile)
nawito          xboard  -270    1906    https://github.com/etorresf/Nawito1906                  (gcc -O3 *.c -o nawito -lm)
neurogrape      uci     -2400   1.1     https://github.com/m00natic/neuroGrape                  (timeout: 0,3s/move)
nino            xboard* -1700   .       https://github.com/allfre2/nino                         (create bin, no name, time: 1s/move)
olipow          xboard  -1250   2.2.3   https://github.com/olithink/olipow2                     (rare loses on time)
pigeon          uci     -1330   1.5.1   https://github.com/StuartRiffle/pigeon                  (cmake, rare ill.move)
plp             uci     -850    .       https://github.com/helgefmi/Poor-little-pinkus          (time: 1s/move)
pulse           uci     -1400   2.0.0   https://github.com/fluxroot/pulse                       (cmake, exits when oponents's time == 0)
quokka          uci*    -1400   2.1     https://github.com/mattbruv/Quokka                      (no name, Linux: -lpthread)
sapeli          uci     -730    1.91    https://github.com/SamuraiDangyo/Sapeli                 (MacOSX: -D__unix__)
spacedog        xb+uci* -800    0.97    https://github.com/thorsilver/SpaceDog                  (cmake, no name)
trappist        uci     -1450   .       https://github.com/flurt/Trappist                                       (ill.moves)
tscp            xboard* -1050   1.8.1c  https://github.com/kalafut/fruitbot/blob/master/tscp    (no name)
tucano          xb+uci  +250    9.01    https://github.com/alcides-schulz/Tucano                (Compile: tucano_compile.bat)
xadreco         xboard  -1600   5.85    https://github.com/drbeco/xadreco                       (time: ~1,9s/move, MacOSX: remove &)
zetadva         xboard  -1000   0310    https://github.com/smatovic/ZetaDva                     (MacOSX: remove --static)
zoe             xboard* -2000   ?       https://github.com/jes/zoe                              (time ~2s/move)

[Ras]

zevra2

Zevra 2.1.2 is not stable due to a race condition, unfortunately. You'll only see this however with enough games in parallel to load your CPU to 100%. Zevra has a risky design where it starts a search thread and discards UCI input like "go" while search is active instead of buffering input other than isready/stop/quit for later processing. However, after giving its bestmove, the associated status variable may not be propagated in time to the input thread at full CPU load. In this case, Zevra just misses the command. Tested under Linux with c-chess-cli where Zevra becomes unresponsive after 10-30 games on my machine.

[hgm]

Fairy-Max did not qualify? Or N.E.G. 1.2?

What exactly is your definition of 'complete implementation'? Must it support analyze mode? Must it support move exclusion? Must it support the highlight protocol? CECP has lots of optional commands.

I suppose KingSlayer/simple did not qualify because it has no analyze mode?

[unserializable]

Hi, I salute this effort -- while writing Monchester, it took time to find open source code that compiled and performed reasonably. I ended up settling on three regular sparring partners, N.E.G. 1.3, ChessPuter and GiuChess 1.0 -- these were already good because they compiled either out of the box or with very minor modifications. Only ChessPuter never forfeited a game though -- GiuChess 1.0 forfeited when it made draw claims little bit early on near 50-moves or 3-fold repetition situations, N.E.G. had problems getting out of check when only legal move for that was e.p. capture.

What exactly is your definition of 'complete implementation'? Must it support analyze mode? Must it support move exclusion? Must it support the highlight protocol? CECP has lots of optional commands.

I second this query for 'complete implementation' definition and made detailed bug report about N.E.G. 1.3 at http://talkchess.com/forum3/viewtopic.p ... 51#p871551 .

If analyze mode is not considered necessary for complete implementation, then I hope Monchester 1.0 could also enter this honorable list.

[fabianVDW]

FabChess will also meat your criterium, that is if you have a rustc and cargo installed.

[hgm]

I second this query for 'complete implementation' definition and made detailed bug report about N.E.G. 1.3 at http://talkchess.com/forum3/viewtopic.p ... 51#p871551 .

Oops, I completely missed this!

Thanks for reporting. Indeed, N.E.G. doesn't have e.p. capture in its move generator. I figured that this would not be important at its level. So if the only check evasion is an e.p. capture it considers itself checkmated. But since it also doesn't detect checkmate as game end, and relies on the GUI for that, it just plays an illegal move, which it would also do when it was really checkmated.

I will see what I can do about that.

[OliverBr]

Fairy-Max did not qualify? Or N.E.G. 1.2?

Fairy-Max was actually a hot candidate and I was very surprised to see it having crashed. It was version 4.8 with the source from here: https://home.hccnet.nl/h.g.muller/dwnldpage.html

I see now that there is Version 5 still to check out. If you tell me the the link to a reliable source, I will do it. Same for N.E.G.

What exactly is your definition of 'complete implementation'? Must it support analyze mode? Must it support move exclusion? Must it support the highlight protocol? CECP has lots of optional commands.

You are right, my definition wasn't exact enough, so I correct:

3) The engine must implement either UCI or CECP protocols so far that it can play a full tournament with any tourney-manager and an opening book handled by the tourney-manager (like cutechess-cli).

(Thus analyze mode is not necessary, I guess this would kill quite a number of candidates.)

I suppose KingSlayer/simple did not qualify because it has no analyze mode?

I think I haven't tested it yet. If you provide a link to the newest sources, I will check it.
Same goes for any engines I might have been missing. Any tip is welcome!

[OliverBr]

zevra2

Zevra 2.1.2 is not stable due to a race condition, unfortunately. You'll only see this however with enough games in parallel to load your CPU to 100%. Zevra has a risky design where it starts a search thread and discards UCI input like "go" while search is active instead of buffering input other than isready/stop/quit for later processing. However, after giving its bestmove, the associated status variable may not be propagated in time to the input thread at full CPU load. In this case, Zevra just misses the command. Tested under Linux with c-chess-cli where Zevra becomes unresponsive after 10-30 games on my machine.

This is strange, because all tests I did showed Zevra 2.1.2 very stable. Computer is a Debian Linux, AMD EPYC 7502P 32-Core Processor. A snapshot during the tournament:

Code: Select all

    # PLAYER               :  RATING  ERROR  POINTS  PLAYED   (%)     W    D     L  D(%)  CFS(%)
   1 Zevra v2.1.2 r248    :     285     18  1475.0    1767  83.5  1377  196   194  11.1     100
   2 OliThink 4.1.3       :       0   ----   292.0    1767  16.5   194  196  1377  11.1     ---

White advantage = 22.53 +/- 10.35
Draw rate (equal opponents) = 15.98 % +/- 1.03

And this was the call (parameters replaced)

Code: Select all

./cutechess-cli  \
-engine cmd=zevra2 proto=uci \
-engine cmd=olithink413 proto=xboard \
-each tc=40/10 dir=bin \
-games 2 -rounds 2500 -concurrency 32 \
-openings file=openings/book_3moves_cp25-49_13580pos.pgn -repeat \
-pgnout tourn.pgn min fi

[OliverBr]

Hi, I salute this effort -- while writing Monchester, it took time to find open source code that compiled and performed reasonably. I ended up settling on three regular sparring partners, N.E.G. 1.3, ChessPuter and GiuChess 1.0 -- these were already good because they compiled either out of the box or with very minor modifications. Only ChessPuter never forfeited a game though -- GiuChess 1.0 forfeited when it made draw claims little bit early on near 50-moves or 3-fold repetition situations, N.E.G. had problems getting out of check when only legal move for that was e.p. capture.

If analyze mode is not considered necessary for complete implementation, then I hope Monchester 1.0 could also enter this honorable list.

I think I have tested ChessPuter and GiuChess, but maybe it were the wrong versions. Please provide me a link to the source to every engine, including Monchester and I will retest it. Thank you very much.