c-chess-cli

Discussion of chess software programming and technical issues.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
lucasart
Posts: 3168
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: c-chess-cli

Post by lucasart » Sat Oct 31, 2020 12:24 am

lucasart wrote:
Mon Oct 26, 2020 12:21 am
lucasart wrote:
Thu Oct 01, 2020 12:02 am
3/ no tournaments yet. just started working on it. first i need to revamp the command line parsing logic to replicate that of cutechess-cli. the whole -option value1[:value2], doesn't work well in the context of N engines.
c-chess-cli is now capable of running tournaments. Gauntlets only for now, Round-Robin will follow soon. The syntax is now mostly identical to cutechess-cli. https://github.com/lucasart/c-chess-cli/.
Round-robin is available now.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Ras
Posts: 1620
Joined: Tue Aug 30, 2016 6:19 pm
Full name: Rasmus Althoff
Contact:

Re: c-chess-cli

Post by Ras » Sat Oct 31, 2020 12:42 am

lucasart wrote:
Sat Oct 31, 2020 12:24 am
Round-robin is available now.
Thanks for the update! I like it more every day, really a no-nonsense tool and no dependency hell.

If I may suggest new features:

1) Adjustable log level. I'd like to choose between full log, or errors only. In the latter case, ideally only generating log files upon the first error in a thread. Like "-log errors" and "-log all" or so, possibly with some well-defined default if "-log" is given without log level.

2) Output some sort of final statistics for the search depth of each engine over a match: median depth, average depth, standard deviation. Ideally also when the program is interrupted with CTRL-C, via a signal handler. That would allow checking whether search changes improve the time to depth.
Rasmus Althoff
https://www.ct800.net

User avatar
lucasart
Posts: 3168
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: c-chess-cli

Post by lucasart » Sat Oct 31, 2020 1:30 am

Ras wrote:
Sat Oct 31, 2020 12:42 am
1) Adjustable log level. I'd like to choose between full log, or errors only. In the latter case, ideally only generating log files upon the first error in a thread. Like "-log errors" and "-log all" or so, possibly with some well-defined default if "-log" is given without log level.
Typically errors (defined vaguely as anything that goes wrong) are logged to stdout one way or another:
  • time losses: logged to stdout (and PGN).
  • disconnection: fatal error (c-chess-cli stops), logged to stdout.
  • engine crashes: logged to stdout. this is detected indirectly as an I/O error ("could not read from engine"), as its observable effect is that we get a broken pipe, which we can't read from (since the write end of the pipe was owned by the child process, which was terminated, and therefore the file handle was closed by the OS).
  • illegal move: logged to stdout (and PGN).
  • illegal moves in PV sent by engines: cutechess-cli logs them to stdout. c-chess-cli logs them in per thread log files as 'WARNING' messages, but remains silent on stdout. Perhaps, I should replicate the cutechess-cli behavior here ? Pro: user is forced to see them. Con: very spammy once you have an offending engine...
But I see your point. Unless you are looking at stdout the whole time, this is not practial. My bash-fu is somewhat limited, but perhaps there is a way to duplicate stdout (ie. display it in the terminal and write it to a file at the same time). Still, you'd have to grep that, and you're not sure what you should be grep'ing for, since the various problems are not tagged consistently.

This would be an interesting feature indeed (which cutechess-cli is also missing), but requires some thoughts on how to do it right.
Ras wrote:
Sat Oct 31, 2020 12:42 am
2) Output some sort of final statistics for the search depth of each engine over a match: median depth, average depth, standard deviation. Ideally also when the program is interrupted with CTRL-C, via a signal handler. That would allow checking whether search changes improve the time to depth.
Also intersting. But first, I would like to bring c-chess-cli on par with cutechess-cli, by writing `{score/depth time}` PGN comments, and adding some of the PGN tags that I find useful. This will pave the way, as I have a Game object which would store the information.

Catching Ctrl+C, in a proper way (ie. passes clang's thread/memory/leak sanitizers), will be an interesting programming exercise.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
lucasart
Posts: 3168
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: c-chess-cli

Post by lucasart » Sat Oct 31, 2020 11:06 pm

lucasart wrote:
Sat Oct 31, 2020 1:30 am
first, I would like to bring c-chess-cli on par with cutechess-cli, by writing `{score/depth time}` PGN comments
Is there any standard for PGN comments ?

ChessGUI (from Graham's tournament) produces this:

Code: Select all

19.Rfd1 {(Rfd1) [%eval 37,29] [%emt 00:00:32]} Rad8 {(Rfd8) [%eval -40,21] [%emt 00:01:06]}
Cutechess produces this:

Code: Select all

37. Kxe2 {-267.46/1 0s} Rf3 {-267.46/1 0s}
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

rongle99
Posts: 3
Joined: Mon Aug 03, 2020 6:00 am
Full name: Ibrahim Adam

Re: c-chess-cli

Post by rongle99 » Sun Nov 01, 2020 6:34 am

could you please include NPM report in PGN like this: +1.2/22 1.5s 1590352

1590352 = NPM

thus we can get the NPS report for each engine

User avatar
lucasart
Posts: 3168
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: c-chess-cli

Post by lucasart » Sun Nov 01, 2020 10:34 am

rongle99 wrote:
Sun Nov 01, 2020 6:34 am
could you please include NPM report in PGN like this: +1.2/22 1.5s 1590352

1590352 = NPM

thus we can get the NPS report for each engine
Does this correspond to a standard ?

If not, is there a program (with a meaningful user base), that parses PGN comments of the form '{score/depth time nodes}' ?
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

AndrewGrant
Posts: 876
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: c-chess-cli

Post by AndrewGrant » Sun Nov 01, 2020 5:08 pm

Personally, I don't think that sort of information has much place in the PGN file. Its the sort of thing I strip out of the PGN file the moment I create it, because smaller PGNs are faster to process.

abulmo2
Posts: 288
Joined: Fri Dec 16, 2016 10:04 am
Contact:

Re: c-chess-cli

Post by abulmo2 » Sun Nov 01, 2020 5:58 pm

lucasart wrote:
Sat Oct 31, 2020 11:06 pm
Cutechess produces this:

Code: Select all

37. Kxe2 {-267.46/1 0s} Rf3 {-267.46/1 0s}
I prefer this one. Xboard/winboard also produces something similar.
Richard Delorme

rongle99
Posts: 3
Joined: Mon Aug 03, 2020 6:00 am
Full name: Ibrahim Adam

Re: c-chess-cli

Post by rongle99 » Mon Nov 02, 2020 3:43 am

the author of BanksiaGui said that this format (+1.2/22 1.5s npm) is a standard, and he applied it in Banksia GUI

getting reports on engine speed is very important!

or if it is not added to the PGN file, then the statistics regarding NPS & NPM must be reported in a separate .txt file like this => http://prntscr.com/vbj4ns

User avatar
lucasart
Posts: 3168
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: c-chess-cli

Post by lucasart » Mon Nov 02, 2020 10:44 am

AndrewGrant wrote:
Sun Nov 01, 2020 5:08 pm
Personally, I don't think that sort of information has much place in the PGN file. Its the sort of thing I strip out of the PGN file the moment I create it, because smaller PGNs are faster to process.
If not in the PGN, where else? You want to produce 2 files?

cutechess has an option to generate clean PGN. of course I will replicate it.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Post Reply