Cutechess-cli 0.3.0 released

[ilari]

Hi,
The problem is that Aristarch doesn't tell that it can play standard chess. It lists "nocastle" as it's only supported game type. If it really can play standard chess it should send "feature variants="normal,nocastle"". If it didn't send any "variants" info, then cutechess-cli would assume that it only supports standard chess.

[bhlangonijr]

Thank you for your efforts Ilari. I am already using the new version.

[ilari]

Thank you for your efforts Ilari. I am already using the new version.

Thanks. v0.3.0 is the first version with no known bugs, so we must be doing something right. I hope the next one will be as stable.

[Andres Valverde]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine "Rybka v2.2n2.mp.w32.exe"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is ", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

Hi, I can find the binaries for cutechess-cli, but not for the GUI...

[ilari]

Hi, I can find the binaries for cutechess-cli, but not for the GUI...

That's because there are no binaries for the GUI yet. It's still at a pre-alpha stage, and not that useful for end-users. The good news is that we're slowly shifting our focus to the GUI, and some cool things are happening. More about that later...

[Andres Valverde]

Hi, I can find the binaries for cutechess-cli, but not for the GUI...

That's because there are no binaries for the GUI yet. It's still at a pre-alpha stage, and not that useful for end-users. The good news is that we're slowly shifting our focus to the GUI, and some cool things are happening. More about that later...

Great news, I'm a daily user of cutechess-cli and it is an extremely useful tool !

[Engin]

hello,
first of all many thx for this nice tool, after i testing some games with my own Tornado seems that its not use is own book, i look at the pgn games and its analyse on move 1 every game.

i set the option.OwnBook=true but this seems not work for UCI engines

other strange is on that the engines using only half of the speed/core, does they using ponder too with 1 cpu ??

on my dual core i see in the task manager that both programm using only 25% so both are using 50 % = 1 cpu, i have 2 cores why they not using 50% vs 50 % ?

does it using only polyglot created book or pgn book ? no own book allowed?

and of course is more nice with real GUI who we are see what is going on the board, even they are faster games.

i am using mostly the shredder GUI at first because i can see what is going on the board and the results with elo differences too.

[ilari]

hello,
first of all many thx for this nice tool, after i testing some games with my own Tornado seems that its not use is own book, i look at the pgn games and its analyse on move 1 every game.

i set the option.OwnBook=true but this seems not work for UCI engines

Could you paste the exact command that you used here? option.OwnBook=true should work just fine, you can verify that it's sent to the engine by running cutechess-cli with the -debug option.

I think Tornado won't find its book if you run it from a different folder without setting the "dir" option in cutechess-cli. Try something like "cutechess-cli -fcp dir=path_to_tornado cmd=tornado.exe proto=uci ...".

other strange is on that the engines using only half of the speed/core, does they using ponder too with 1 cpu ??

on my dual core i see in the task manager that both programm using only 25% so both are using 50 % = 1 cpu, i have 2 cores why they not using 50% vs 50 % ?

Cutechess-cli doesn't support pondering yet, and if you're running single-cpu engines then it's expected that only 1 core is fully utilized. If you want to use all your cores you can set -concurrency=2 so that two games are played at the same time.

does it using only polyglot created book or pgn book ? no own book allowed?

Both Polyglot books and PGN books can be used, even at the same time. And own books are of course allowed. I don't even know how an interface could prevent them from being used.

and of course is more nice with real GUI who we are see what is going on the board, even they are faster games.

i am using mostly the shredder GUI at first because i can see what is going on the board and the results with elo differences too.

Of course, but that's a different project, called cutechess-gui. Cutechess-cli is GUI-less by design so that it would be fast, small, highly portable, and not dependent on an xserver.

[Ferdy]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine "Rybka v2.2n2.mp.w32.exe"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is ", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

Hi Ilari,

Can you have a look on this, I got a strange adjudication today in 2 (games below) of the 62 games I played vs arasan 12.3. My options are -resign 5 900, -concurrency 5, -wait 5, -draw 80 1. Hardware is amd x6, OS win 7, 64bit. Other adjudications are fine. I use cutechess-cli 0.3.0. I have not yet checked other games vs other engines.

sample #1: adjudicated at +0.24

Code: Select all

[Event "?"]
[Site "?"]
[Date "2011.05.27"]
[Round "21"]
[White "Deuterium v11.01.28.159"]
[Black "Arasan 12.3 (1 cpu)"]
[Result "1-0"]
[BlackTimeControl "40/15"]
[PlyCount "47"]
[Termination "adjudication"]
[WhiteTimeControl "40/10"]

1. e4 {book} d6 {book} 2. d4 {book} Nf6 {book} 3. Nc3 {book} g6 {book}
4. Nf3 {book} Bg7 {book} 5. Be2 {book} O-O {book} 6. O-O {book} c6 {book}
7. h3 {+0.87/8 0.28s} d5 {-0.40/10 1.3s} 8. e5 {+0.81/9 0.28s}
Ne8 {-0.84/8 0.36s} 9. Bd3 {+1.00/10 0.27s} f6 {-0.80/8 0.27s}
10. exf6 {+1.08/10 0.28s} exf6 {-0.60/8 0.37s} 11. Re1 {+0.79/10 0.28s}
Na6 {-0.59/8 0.39s} 12. a3 {+0.64/10 0.28s} Nd6 {-0.44/8 0.36s}
13. Bf4 {+0.73/10 0.20s} Nc7 {-0.32/7 0.12s} 14. b3 {+0.74/8 0.28s}
b6 {-0.38/7 0.27s} 15. Be3 {+0.52/9 0.28s} Re8 {-0.12/8 0.37s}
16. Re2 {+0.37/10 0.28s} Qd7 {+0.04/8 0.36s} 17. Qf1 {+0.46/9 0.28s}
Bb7 {+0.13/8 0.25s} 18. Bf4 {+0.46/7 0.28s} g5 {+0.20/8 0.34s}
19. Rxe8+ {+0.33/9 0.28s} Rxe8 {+0.29/10 0.34s} 20. Bc1 {+0.09/10 0.28s}
Ne4 {+0.48/7 0.11s} 21. Bb2 {+0.19/10 0.28s} Bf8 {+0.40/7 0.36s}
22. Qe1 {+0.08/8 0.28s} c5 {+0.33/7 0.36s} 23. a4 {+0.38/7 0.28s}
Bd6 {+0.52/6 0.25s} 24. Nb5 {+0.24/7 0.22s, White wins by adjudication} 1-0

sample #2: adjudicated at +1.55

Code: Select all

[Event "?"]
[Site "?"]
[Date "2011.05.27"]
[Round "35"]
[White "Deuterium v11.01.28.159"]
[Black "Arasan 12.3 (1 cpu)"]
[Result "1-0"]
[BlackTimeControl "40/15"]
[PlyCount "31"]
[Termination "adjudication"]
[WhiteTimeControl "40/10"]

1. e4 {book} c5 {book} 2. Nf3 {book} Nc6 {book} 3. d4 {book} cxd4 {book}
4. Nxd4 {book} Nf6 {book} 5. Nc3 {book} e5 {book} 6. Ndb5 {book} d6 {book}
7. Bg5 {book} a6 {book} 8. Na3 {book} b5 {book} 9. Bxf6 {book} gxf6 {book}
10. Nd5 {book} f5 {book} 11. Bd3 {+0.44/10 0.30s} Be6 {-0.60/10 1.4s}
12. O-O {+0.38/10 0.31s} Bh6 {-0.83/8 0.41s} 13. Qh5 {+0.68/9 0.31s}
Bg5 {-1.09/8 0.41s} 14. f4 {+0.77/9 0.31s} Bh4 {-0.96/7 0.28s}
15. Qh6 {+0.93/8 0.31s} Kd7 {-1.30/6 0.42s}
16. c4 {+1.55/8 0.31s, White wins by adjudication} 1-0

[Ferdy]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine "Rybka v2.2n2.mp.w32.exe"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is ", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

Hi Ilari,

Can you have a look on this, I got a strange adjudication today in 2 (games below) of the 62 games I played vs arasan 12.3. My options are -resign 5 900, -concurrency 5, -wait 5, -draw 80 1. Hardware is amd x6, OS win 7, 64bit. Other adjudications are fine. I use cutechess-cli 0.3.0. I have not yet checked other games vs other engines.

sample #1: adjudicated at +0.24

Code: Select all

[Event "?"]
[Site "?"]
[Date "2011.05.27"]
[Round "21"]
[White "Deuterium v11.01.28.159"]
[Black "Arasan 12.3 (1 cpu)"]
[Result "1-0"]
[BlackTimeControl "40/15"]
[PlyCount "47"]
[Termination "adjudication"]
[WhiteTimeControl "40/10"]

1. e4 {book} d6 {book} 2. d4 {book} Nf6 {book} 3. Nc3 {book} g6 {book}
4. Nf3 {book} Bg7 {book} 5. Be2 {book} O-O {book} 6. O-O {book} c6 {book}
7. h3 {+0.87/8 0.28s} d5 {-0.40/10 1.3s} 8. e5 {+0.81/9 0.28s}
Ne8 {-0.84/8 0.36s} 9. Bd3 {+1.00/10 0.27s} f6 {-0.80/8 0.27s}
10. exf6 {+1.08/10 0.28s} exf6 {-0.60/8 0.37s} 11. Re1 {+0.79/10 0.28s}
Na6 {-0.59/8 0.39s} 12. a3 {+0.64/10 0.28s} Nd6 {-0.44/8 0.36s}
13. Bf4 {+0.73/10 0.20s} Nc7 {-0.32/7 0.12s} 14. b3 {+0.74/8 0.28s}
b6 {-0.38/7 0.27s} 15. Be3 {+0.52/9 0.28s} Re8 {-0.12/8 0.37s}
16. Re2 {+0.37/10 0.28s} Qd7 {+0.04/8 0.36s} 17. Qf1 {+0.46/9 0.28s}
Bb7 {+0.13/8 0.25s} 18. Bf4 {+0.46/7 0.28s} g5 {+0.20/8 0.34s}
19. Rxe8+ {+0.33/9 0.28s} Rxe8 {+0.29/10 0.34s} 20. Bc1 {+0.09/10 0.28s}
Ne4 {+0.48/7 0.11s} 21. Bb2 {+0.19/10 0.28s} Bf8 {+0.40/7 0.36s}
22. Qe1 {+0.08/8 0.28s} c5 {+0.33/7 0.36s} 23. a4 {+0.38/7 0.28s}
Bd6 {+0.52/6 0.25s} 24. Nb5 {+0.24/7 0.22s, White wins by adjudication} 1-0

sample #2: adjudicated at +1.55

Code: Select all

[Event "?"]
[Site "?"]
[Date "2011.05.27"]
[Round "35"]
[White "Deuterium v11.01.28.159"]
[Black "Arasan 12.3 (1 cpu)"]
[Result "1-0"]
[BlackTimeControl "40/15"]
[PlyCount "31"]
[Termination "adjudication"]
[WhiteTimeControl "40/10"]

1. e4 {book} c5 {book} 2. Nf3 {book} Nc6 {book} 3. d4 {book} cxd4 {book}
4. Nxd4 {book} Nf6 {book} 5. Nc3 {book} e5 {book} 6. Ndb5 {book} d6 {book}
7. Bg5 {book} a6 {book} 8. Na3 {book} b5 {book} 9. Bxf6 {book} gxf6 {book}
10. Nd5 {book} f5 {book} 11. Bd3 {+0.44/10 0.30s} Be6 {-0.60/10 1.4s}
12. O-O {+0.38/10 0.31s} Bh6 {-0.83/8 0.41s} 13. Qh5 {+0.68/9 0.31s}
Bg5 {-1.09/8 0.41s} 14. f4 {+0.77/9 0.31s} Bh4 {-0.96/7 0.28s}
15. Qh6 {+0.93/8 0.31s} Kd7 {-1.30/6 0.42s}
16. c4 {+1.55/8 0.31s, White wins by adjudication} 1-0

I do a second run, same condition and I got 1/100 games, with strange adjudication.

Adjudicated at -1.15.

Code: Select all

[Event "?"]
[Site "?"]
[Date "2011.05.27"]
[Round "98"]
[White "Arasan 12.3 (1 cpu)"]
[Black "Deuterium v11.01.28.159"]
[Result "0-1"]
[BlackTimeControl "40/10"]
[PlyCount "46"]
[Termination "adjudication"]
[WhiteTimeControl "40/15"]

1. d4 {book} Nf6 {book} 2. c4 {book} g6 {book} 3. Nc3 {book} Bg7 {book}
4. e4 {book} d6 {book} 5. f3 {book} O-O {book} 6. Be3 {book} e5 {book}
7. Nge2 {book} c6 {book} 8. Qb3 {-0.04/6 0.37s} exd4 {-0.19/9 0.30s}
9. Nxd4 {+0.12/9 0.36s} Nbd7 {-0.64/9 0.28s} 10. Qa3 {+0.20/8 0.34s}
Qe7 {-0.59/8 0.22s} 11. Be2 {+0.24/8 0.36s} Ne5 {-0.61/10 0.28s}
12. O-O-O {+0.24/8 0.37s} Bd7 {-0.50/9 0.28s} 13. g4 {+0.28/8 0.39s}
b5 {-0.06/8 0.28s} 14. cxb5 {+1.02/7 0.32s} c5 {-0.49/8 0.28s}
15. Nc2 {+1.04/8 0.34s} Be6 {-1.10/9 0.28s} 16. Qa6 {+1.00/8 0.38s}
Rfd8 {-0.43/7 0.28s} 17. Kb1 {+0.96/8 0.38s} d5 {-0.84/9 0.28s}
18. g5 {+1.44/6 0.34s} Nxe4 {-0.97/10 0.30s} 19. fxe4 {+1.32/9 0.33s}
d4 {-0.96/10 0.27s} 20. Nxd4 {+1.32/8 0.30s} cxd4 {-1.15/9 0.032s}
21. Rxd4 {+1.29/9 0.33s} Rxd4 {-0.79/8 0.047s} 22. Bxd4 {+1.84/8 0.28s}
Qxg5 {-1.15/8 0.31s} 23. b6 {+1.90/8 0.30s}
Qe7 {-1.15/8 0.28s, Black wins by adjudication} 0-1

BTW just want to verify, how cutechess-cli adjudicate if I use -resign 5 900? Do the 2 engines have to agree on the score of 900 and count of 5?

I just finished testing vs etchess, and in 100 games adjudications are all right .