c-chess-cli

[lucasart]

3/ no tournaments yet. just started working on it. first i need to revamp the command line parsing logic to replicate that of cutechess-cli. the whole -option value1[:value2], doesn't work well in the context of N engines.

c-chess-cli is now capable of running tournaments. Gauntlets only for now, Round-Robin will follow soon. The syntax is now mostly identical to cutechess-cli. https://github.com/lucasart/c-chess-cli/.

Round-robin is available now.

[Ras]

Round-robin is available now.

Thanks for the update! I like it more every day, really a no-nonsense tool and no dependency hell.

If I may suggest new features:

1) Adjustable log level. I'd like to choose between full log, or errors only. In the latter case, ideally only generating log files upon the first error in a thread. Like "-log errors" and "-log all" or so, possibly with some well-defined default if "-log" is given without log level.

2) Output some sort of final statistics for the search depth of each engine over a match: median depth, average depth, standard deviation. Ideally also when the program is interrupted with CTRL-C, via a signal handler. That would allow checking whether search changes improve the time to depth.

[lucasart]

1) Adjustable log level. I'd like to choose between full log, or errors only. In the latter case, ideally only generating log files upon the first error in a thread. Like "-log errors" and "-log all" or so, possibly with some well-defined default if "-log" is given without log level.

Typically errors (defined vaguely as anything that goes wrong) are logged to stdout one way or another:

• time losses: logged to stdout (and PGN).
• disconnection: fatal error (c-chess-cli stops), logged to stdout.
• engine crashes: logged to stdout. this is detected indirectly as an I/O error ("could not read from engine"), as its observable effect is that we get a broken pipe, which we can't read from (since the write end of the pipe was owned by the child process, which was terminated, and therefore the file handle was closed by the OS).
• illegal move: logged to stdout (and PGN).
• illegal moves in PV sent by engines: cutechess-cli logs them to stdout. c-chess-cli logs them in per thread log files as 'WARNING' messages, but remains silent on stdout. Perhaps, I should replicate the cutechess-cli behavior here ? Pro: user is forced to see them. Con: very spammy once you have an offending engine...

But I see your point. Unless you are looking at stdout the whole time, this is not practial. My bash-fu is somewhat limited, but perhaps there is a way to duplicate stdout (ie. display it in the terminal and write it to a file at the same time). Still, you'd have to grep that, and you're not sure what you should be grep'ing for, since the various problems are not tagged consistently.

This would be an interesting feature indeed (which cutechess-cli is also missing), but requires some thoughts on how to do it right.

2) Output some sort of final statistics for the search depth of each engine over a match: median depth, average depth, standard deviation. Ideally also when the program is interrupted with CTRL-C, via a signal handler. That would allow checking whether search changes improve the time to depth.

Also intersting. But first, I would like to bring c-chess-cli on par with cutechess-cli, by writing `{score/depth time}` PGN comments, and adding some of the PGN tags that I find useful. This will pave the way, as I have a Game object which would store the information.

Catching Ctrl+C, in a proper way (ie. passes clang's thread/memory/leak sanitizers), will be an interesting programming exercise.

[lucasart]

first, I would like to bring c-chess-cli on par with cutechess-cli, by writing `{score/depth time}` PGN comments

Is there any standard for PGN comments ?

ChessGUI (from Graham's tournament) produces this:

Code: Select all

19.Rfd1 {(Rfd1) [%eval 37,29] [%emt 00:00:32]} Rad8 {(Rfd8) [%eval -40,21] [%emt 00:01:06]}

Cutechess produces this:

Code: Select all

37. Kxe2 {-267.46/1 0s} Rf3 {-267.46/1 0s}

[rongle99]

could you please include NPM report in PGN like this: +1.2/22 1.5s 1590352

1590352 = NPM

thus we can get the NPS report for each engine

[lucasart]

could you please include NPM report in PGN like this: +1.2/22 1.5s 1590352

1590352 = NPM

thus we can get the NPS report for each engine

Does this correspond to a standard ?

If not, is there a program (with a meaningful user base), that parses PGN comments of the form '{score/depth time nodes}' ?

[AndrewGrant]

Personally, I don't think that sort of information has much place in the PGN file. Its the sort of thing I strip out of the PGN file the moment I create it, because smaller PGNs are faster to process.

[abulmo2]

Cutechess produces this:

Code: Select all

37. Kxe2 {-267.46/1 0s} Rf3 {-267.46/1 0s}

I prefer this one. Xboard/winboard also produces something similar.

[rongle99]

the author of BanksiaGui said that this format (+1.2/22 1.5s npm) is a standard, and he applied it in Banksia GUI

getting reports on engine speed is very important!

or if it is not added to the PGN file, then the statistics regarding NPS & NPM must be reported in a separate .txt file like this => http://prntscr.com/vbj4ns

[lucasart]

Personally, I don't think that sort of information has much place in the PGN file. Its the sort of thing I strip out of the PGN file the moment I create it, because smaller PGNs are faster to process.

If not in the PGN, where else? You want to produce 2 files?

cutechess has an option to generate clean PGN. of course I will replicate it.