List of bugfree, opensource Linux and MacOSX engines

[OliverBr]

EDIT: Ah, sorry, it crashed again after about 500 games and it's reproducible:

Code: Select all

Terminating process of engine Fairy-Max 4.8L(24)
Finished game 496 (OliThink 5.9.0 vs Fairy-Max 4.8L): 1-0 {Black's connection stalls}

You have the game, so I can check?

Here is another one:

Code: Select all

Terminating process of engine Fairy-Max 4.8L(7)
Finished game 1193 (Fairy-Max 4.8L vs OliThink 5.9.0): 0-1 {White's connection stalls}

[pgn]
[Event "?"]
[Site "?"]
[Date "2020.11.10"]
[Round "597"]
[White "Fairy-Max 4.8L"]
[Black "OliThink 5.9.0"]
[Result "0-1"]
[ECO "A00"]
[GameDuration "00:00:35"]
[GameEndTime "2020-11-10T21:41:45.495 CET"]
[GameStartTime "2020-11-10T21:41:10.313 CET"]
[Opening "Anderssen's Opening"]
[PlyCount "64"]
[Termination "stalled connection"]
[TimeControl "40/12"]

1. a3 {book} d6 {book} 2. d3 {book} b6 {book} 3. e3 {book} e5 {book}
4. Be2 {0.00/6 0.26s} Nf6 {-0.12/14 0.52s} 5. Nf3 {+0.14/7 0.24s}
Be7 {-0.10/15 0.28s} 6. O-O {+0.03/7 0.28s} a5 {-0.06/15 0.21s}
7. h4 {-0.18/6 0.24s} O-O {+0.30/14 0.41s} 8. Nc3 {-0.03/7 0.34s}
Bd7 {+0.31/15 0.26s} 9. d4 {-0.13/8 0.62s} e4 {+0.49/15 0.26s}
10. Nd2 {-0.11/8 0.29s} d5 {+0.60/16 0.20s} 11. f3 {-0.06/8 0.54s}
exf3 {+0.59/15 0.20s} 12. Nxf3 {-0.09/6 0.17s} Bd6 {+0.65/15 0.23s}
13. Ne5 {-0.05/6 0.18s} c6 {+0.51/15 0.41s} 14. Nxd7 {+0.19/7 0.50s}
Nbxd7 {+0.68/15 0.22s} 15. Bd2 {+0.37/6 0.18s} Bg3 {+0.64/14 0.30s}
16. Be1 {+0.18/7 0.20s} Bc7 {+0.69/14 0.23s} 17. b4 {+0.30/6 0.37s}
Qe7 {+1.05/15 0.42s} 18. Rf3 {+0.28/6 0.15s} axb4 {+1.04/16 0.48s}
19. axb4 {+0.22/7 0.21s} b5 {+1.02/17 0.32s} 20. Rxa8 {+0.17/7 0.38s}
Rxa8 {+1.02/17 0.21s} 21. Qb1 {+0.05/7 0.38s} Ng4 {+1.02/17 0.28s}
22. Nd1 {+0.02/6 0.15s} Ndf6 {+1.05/15 0.33s} 23. Bg3 {+0.27/7 0.22s}
Ne4 {+1.25/16 0.27s} 24. Bxc7 {+0.23/9 0.76s} Qxc7 {+1.25/16 0.25s}
25. Rh3 {+0.24/9 0.27s} h5 {+1.25/17 0.25s} 26. Qb3 {+0.36/6 0.15s}
Ra1 {+2.88/17 0.21s} 27. Qb2 {-0.36/9 0.25s} Qa7 {+2.67/19 0.22s}
28. Rf3 {-1.39/9 0.24s} Qa4 {+2.79/19 0.22s} 29. Rf1 {-1.58/8 0.14s}
Ng3 {+3.01/20 0.23s} 30. Nc3 {-1.84/9 0.19s} Nxe2+ {+3.16/21 0.28s}
31. Nxe2 {-2.48/12 0.19s} Rxf1+ {+3.04/22 0.30s} 32. Kxf1 {-2.00/14 1.2s}
Nxe3+ {+3.16/22 0.41s, White's connection stalls} 0-1
[/pgn]

[OliverBr]

This should be cured in version 5, which invalidates its hash table before any search where the use of null move was switched.

The latest source of Fairy-Max can be found in my git repository:

http://hgm.nubati.net/cgi-bin/gitweb.cgi

Just download the latest snapshot of the main branch. Same for KingSlayer / simple.

Unfortunately there are some issues with Fairy-Max 5, too. No invalid moves, some timeouts (much less than 4.8) and also sometimes a crash:

Code: Select all

Terminating process of engine Fairy-Max 5.0b(43)
Finished game 759 (Fairy-Max 5.0b vs OliThink 4.1.3): 0-1 {White's connection stalls}

From the frequency it looks like the same bug as in 4.8.
If you need a game here, too, I will rerun cutechess-cli without the "fi" option.

[sedicla]

Hi Oliver

Code: Select all

tucano          xb+uci  +250    9.01    https://github.com/alcides-schulz/Tucano                (Compile: tucano_compile.bat)

Seems tucano is just missing a makefile? I have that on the todo list, I will add and let you know. If there's anything else, please let me know I'll take a look.

thanks.

[abulmo2]

Hello
I checked a lot of open source engines, if they met the following criterias:

1) Open source (most are on git anyway)
2) Easy compile with make/cmake on both, Linux and MacOSX, with g++ and clang, telling their name on init.
3) Complete implementation of either UCI or CECP protocol
4) Stable over thousands of games
5) Bugfree in terms of timeouts and illegal moves and draw repitions.

From over 1000 engines I found, only 48 met all the criterias.
This is the list, the third column is the relative strength when playing h2h against OliThink 5.9.0 on very fast time control:

From your other posts it seems you are using cutechess-cli. My own engines (amoeba) do illegal moves according to cutechess-cli, except that when I look at the log produced by them, cutechess-cli is clearly at fault.
Here is an example of game with illegal move:

Code: Select all

[Event "rr-4.0"]
[Site "?"]
[Date "2020.11.11"]
[Round "8"]
[White "Amoeba 4.0.17b.l64p-l"]
[Black "Amoeba 4.0.5.l64p-l"]
[Result "0-1"]
[ECO "A58"]
[GameDuration "00:00:00"]
[GameEndTime "2020-11-11T07:29:10.774 CET"]
[GameStartTime "2020-11-11T07:29:10.455 CET"]
[Opening "Benko gambit accepted"]
[PlyCount "12"]
[Termination "illegal move"]
[TimeControl "10+0.1"]

1. d4 {book} Nf6 {book} 2. c4 {book} c5 {book} 3. d5 {book} b5 {book}
4. cxb5 {book} a6 {book} 5. bxa6 {book} g6 {book} 6. Nc3 {book}
Bxa6 {book, White makes an illegal move: d8a5} 0-1

And the corresponding log from Amoeba 4.0.17b:

Code: Select all

[2020.11.11 7:29:10.091] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.091] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.102] Amoeba 4.0.17b.l64p-l< ucinewgame
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< setoption name Ponder value false
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< position startpos
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3 c8a6
[2020.11.11 7:29:10.444] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.444] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.456] Amoeba 4.0.17b.l64p-l< ucinewgame
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< setoption name Ponder value false
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l< go wtime 10100 btime 10100 winc 100 binc 100
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# time> 10.1/0+0.1 -> time.min0.234412, time.max: 0.334875, time.extra: 0.66975
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# search options: Option(Time(0.234412, 0.334875, 0.66975), Nodes(18446744073709551615), Depth(0, 127), Score(0), CPU(16, CPUAffinity(0, 0)), 1, true, false, true)
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# smp> task[0] launched
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# smp > task[0] iterate
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l> info depth 1 seldepth 1 score cp -16 time 0 nodes 24 nps 95200 pv c8a6 
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l> info depth 2 seldepth 4 score cp -34 time 0 nodes 98 nps 297962 pv c8a6 c1f4 
[...]
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l> info depth 13 seldepth 28 score cp -53 time 307 nodes 561445 nps 1824714 pv d8a5 g1f3 f6e4 c1d2 e4d2 d1d2 c8a6 e2e4 f8g7 f1a6 a5a6 e4e5 d7d6 e1c1 
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l# smp> task[0] finished: info depth 13 seldepth 28 score cp -53 time 307 nodes 561445 nps 1824714 pv d8a5 g1f3 f6e4 c1d2 e4d2 d1d2 c8a6 e2e4 f8g7 f1a6 a5a6 e4e5 d7d6 e1c1 
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l# smp> task[0] terminated
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l> bestmove d8a5
[2020.11.11 7:29:10.775] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.775] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.787] Amoeba 4.0.17b.l64p-l< ucinewgame

My engine received twice the opening, the second time without the last move and of course give a bestmove from the last position it received. Garbaage in, garbage out.
I am afraid that cutechess-cli is often the one at fault in your tests, not the engines you analysed.

[RubiChess]

I also want to make it on your list.

With latest commit Rubi should compile and run in MacOS. I don't have a Mac but I could compile and run it in a VM.
If you still have problems with latest master, please report here: https://github.com/Matthies/RubiChess/issues/248

Regards, Andreas

[Guenther]

...

From your other posts it seems you are using cutechess-cli. My own engines (amoeba) do illegal moves according to cutechess-cli, except that when I look at the log produced by them, cutechess-cli is clearly at fault.
Here is an example of game with illegal move:

Code: Select all

[Event "rr-4.0"]
[Site "?"]
[Date "2020.11.11"]
[Round "8"]
[White "Amoeba 4.0.17b.l64p-l"]
[Black "Amoeba 4.0.5.l64p-l"]
[Result "0-1"]
[ECO "A58"]
[GameDuration "00:00:00"]
[GameEndTime "2020-11-11T07:29:10.774 CET"]
[GameStartTime "2020-11-11T07:29:10.455 CET"]
[Opening "Benko gambit accepted"]
[PlyCount "12"]
[Termination "illegal move"]
[TimeControl "10+0.1"]

1. d4 {book} Nf6 {book} 2. c4 {book} c5 {book} 3. d5 {book} b5 {book}
4. cxb5 {book} a6 {book} 5. bxa6 {book} g6 {book} 6. Nc3 {book}
Bxa6 {book, White makes an illegal move: d8a5} 0-1

And the corresponding log from Amoeba 4.0.17b:
[2020.11.11 7:29:10.091] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.091] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.102] Amoeba 4.0.17b.l64p-l< ucinewgame
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< setoption name Ponder value false
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< position startpos
[2020.11.11 7:29:10.111] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4
...
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3
[2020.11.11 7:29:10.112] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3 c8a6
[2020.11.11 7:29:10.444] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.444] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.456] Amoeba 4.0.17b.l64p-l< ucinewgame
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< setoption name Ponder value false
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4
...
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< position startpos moves d2d4 g8f6 c2c4 c7c5 d4d5 b7b5 c4b5 a7a6 b5a6 g7g6 b1c3
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.465] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l< go wtime 10100 btime 10100 winc 100 binc 100
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# time> 10.1/0+0.1 -> time.min0.234412, time.max: 0.334875, time.extra: 0.66975
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# search options: Option(Time(0.234412, 0.334875, 0.66975), Nodes(18446744073709551615), Depth(0, 127), Score(0), CPU(16, CPUAffinity(0, 0)), 1, true, false, true)
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# smp> task[0] launched
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l# smp > task[0] iterate
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l> info depth 1 seldepth 1 score cp -16 time 0 nodes 24 nps 95200 pv c8a6
[2020.11.11 7:29:10.467] Amoeba 4.0.17b.l64p-l> info depth 2 seldepth 4 score cp -34 time 0 nodes 98 nps 297962 pv c8a6 c1f4
[...]
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l> info depth 13 seldepth 28 score cp -53 time 307 nodes 561445 nps 1824714 pv d8a5 g1f3 f6e4 c1d2 e4d2 d1d2 c8a6 e2e4 f8g7 f1a6 a5a6 e4e5 d7d6 e1c1
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l# smp> task[0] finished: info depth 13 seldepth 28 score cp -53 time 307 nodes 561445 nps 1824714 pv d8a5 g1f3 f6e4 c1d2 e4d2 d1d2 c8a6 e2e4 f8g7 f1a6 a5a6 e4e5 d7d6 e1c1
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l# smp> task[0] terminated
[2020.11.11 7:29:10.774] Amoeba 4.0.17b.l64p-l> bestmove d8a5
[2020.11.11 7:29:10.775] Amoeba 4.0.17b.l64p-l< isready
[2020.11.11 7:29:10.775] Amoeba 4.0.17b.l64p-l> readyok
[2020.11.11 7:29:10.787] Amoeba 4.0.17b.l64p-l< ucinewgame


My engine received twice the opening, the second time without the last move and of course give a bestmove from the last position it received. Garbaage in, garbage out.
I am afraid that cutechess-cli is often the one at fault in your tests, not the engines you analysed.

I haven't used the cli for long, but never had such a thing in my tests when I used it. (only used pgn start lines though)

What I don't understand from your log is that the opening appears to be sent twice according to Amoebas log.
(Do you have a cli debug file too?)
One time with the missing move and one time without it? BTW is this a pgn or epd or even a polyglot start book used?

[abulmo2]

I am afraid that cutechess-cli is often the one at fault in your tests, not the engines you analysed.

I just look at cutechess's github repository. It has got 150 issues...

[Guenther]

I am afraid that cutechess-cli is often the one at fault in your tests, not the engines you analysed.

I just look at cutechess's github repository. It has got 150 issues... :-(

Those are not all real issues though. A part of them are ... ahem user issues, another part suggestions for enhancements, wishes
and another part GUI issues.

[George Sobala]

I presume you are excluding Stockfish derivatives, though you did not put that into your criteria. Otherwise you could include Cfish and corchess (at least). Also lc0 of course.

[Roland Chastain]

And there is a list of 26 candidates, quite interesting engines, which do compile on both Linux and MacOSX and are stable, but have some flaws, which should be repaired easily.

Code: Select all

hippocampe      xboard* -2150   0.4.2   http://www.computer-chess.org/lib/exe/fetch.php?media=computer_chess:wiki:download:hippocampe-0.4.2.tar.gz (no name, ill.moves)

Hello! I fixed the name issue for Hippocampe, and the issue of the missing promotion suffix. IIRC there is also a problem of illegal castling move (I am not sure, but I believe that the program doesn't check that the castling is not under attack): I will try to fix it later.

hippocampe.zip

Regards.

Roland