Interrupting Cutechess-Cli with CTRL+C yields to unpredictable results.

[OliverBr]

Hello together,

When running a tournament with cutechess-cli and interrupting with "CTRL-C" unpredictable results will follow. I would have expected a forfeit like:

Code: Select all

Finished game 27 (Fruit 2.1 vs OliThink 5.8.6): * {No result}

which happens reliably when killing the background cutechess-cli process with "2".

Code: Select all

killall -2 cutechess-cli

But, when interrupting the foreground process with CTRL-C those can happens, too:

Code: Select all

Finished game 32 (OliThink 5.8.6 vs Fruit 2.1): 1-0 {Black disconnects}
Finished game 4 (OliThink 5.8.6 vs Fruit 2.1): 0-1 {White disconnects}

This shouldn't happen, the result is arbitrary and falsifies the result.

Is this a known bug? Is there a way to avoid this?

[brianr]

Yes, use the newer version with the "fi" parameter.
The one I use says cutechess-cli 1.2.0.
Will have to look for a link, forget where I got it.

[brianr]

https://github.com/cutechess/cutechess/ ... /tag/1.2.0

[OliverBr]

Yes, use the newer version with the "fi" parameter.
The one I use says cutechess-cli 1.2.0.
Will have to look for a link, forget where I got it.

Thank you! This does the trick. I am still using 1.0.0 from 2017 which hasn't go this option yet. Here is the commit:

https://github.com/cutechess/cutechess/ ... 9a8896aada

[OliverBr]

I have installed cutechess-1.2.0 and now it's much better with option -pgnout , but not completely solved.

Running 32 concurrent games, in most cases interrupting with CTRC+C now yields to

Code: Select all

Warning: Omitted incomplete game 58

But still getting 2-3 games which are being rated because "{Black disconnects}".

[brianr]

For timed matches on my 12 core I only run at most 10 games at once.

I also use several other params (like restart and wait).

Issue with all info posted on OliThink GitHub with a 1,000 game match between versions, just FYI.

[OliverBr]

I also use several other params (like restart and wait).

I am not sure about the option "restart=on". Isn't it better to reuse the process and not pre-calculate everything again?

Issue with all info posted on OliThink GitHub with a 1,000 game match between versions, just FYI.

Thank you a very much. This is quite surprising, as my test with this very timecontrol "0.10+1.0" and the exact same opening file yields are completely different result:

Code: Select all

   # PLAYER            :  RATING  ERROR  POINTS  PLAYED   (%)    W    D    L  D(%)  CFS(%)
   1 OliThink 5.8.6    :     183     25   416.5     564  73.8  322  189   53  33.5     100
   2 OliThink 5.7.5    :       0   ----   147.5     564  26.2   53  189  322  33.5     ---

I am not sure how to explain this. Do you have an idea?

[brianr]

Try running with the cutechess command I used in the issue post.

Only 10 of 12 cores used.
Full 6 and select 7 men syzygy adjudication for one.
Restart and wait.

Check your pgn for forfeits on time or other errors if you were using the 32 games at a time results.

Regarding restart on, at one point Lc0 was giving very different match results with and without restart, so I just keep in on all the time now, even for A/B engines. At this time control it does not take any significant time I think.

[Ozymandias]

I am not sure about the option "restart=on".

Code: Select all

restart=MODE		Set the restart mode to MODE which can be:
			'auto': the engine decides whether to restart (default)
			'on': the engine is always restarted between games
			'off': the engine is never restarted between games
			Setting this option does not prevent engines from being
			restarted between rounds in a tournament featuring more
			than two engines.

[OliverBr]

Try running with the cutechess command I used in the issue post.

Only 10 of 12 cores used.
Full 6 and select 7 men syzygy adjudication for one.
Restart and wait.

Where can I download the exact same syzygy tablebases you are using in the curechess-cli command?

Code: Select all

E:/syzygy;E:/syzygy-7