cutechess-cli 0.8.0 released

[Ferdy]

Cutechess-cli is now at version 0.8.2: https://github.com/cutechess/cutechess/ ... /cli-0.8.2

This is a good update for those users that play non-standard variants like Crazyhouse.

There are engines that when I match with my engine cutechess would display the following message.

Code: Select all

Warning: EngineProcess: Destroyed while process is still running.

Is there something to worry about this warning? I have a schedule of 200 games and cutechess keeps on displaying this message around 5 times in 20 games.

If I run a match between my engines, I don't see this message.

[Nordlandia]

Is 6-men adjuction on the priority list?

Potentially it will save alot of time for events such as TCEC.

[ilari]

Cutechess-cli is now at version 0.8.2: https://github.com/cutechess/cutechess/ ... /cli-0.8.2

This is a good update for those users that play non-standard variants like Crazyhouse.

There are engines that when I match with my engine cutechess would display the following message.

Code: Select all

Warning: EngineProcess: Destroyed while process is still running.

Is there something to worry about this warning? I have a schedule of 200 games and cutechess keeps on displaying this message around 5 times in 20 games.

If I run a match between my engines, I don't see this message.

Usually that warning means that cutechess tried to terminate an engine, but the engine process was still alive after the termination timeout. Cutechess should still keep going just fine if that happens. Can you check with Task Manager if there are zombie engine processes still around after you see this warning? And is there some specific free engine with which I could reproduce this warning?

[ilari]

Is 6-men adjuction on the priority list?

Potentially it will save alot of time for events such as TCEC.

Since there are no 6-men Gaviota tbs, it's not so trivial. But we plan on replacing Gaviota with Syzygy which would bring 6-men tbs: https://github.com/cutechess/cutechess/issues/68

It's going to happen after the first GUI release which is hopefully soon.

[Ferdy]

Cutechess-cli is now at version 0.8.2: https://github.com/cutechess/cutechess/ ... /cli-0.8.2

This is a good update for those users that play non-standard variants like Crazyhouse.

There are engines that when I match with my engine cutechess would display the following message.

Code: Select all

Warning: EngineProcess: Destroyed while process is still running.

Is there something to worry about this warning? I have a schedule of 200 games and cutechess keeps on displaying this message around 5 times in 20 games.

If I run a match between my engines, I don't see this message.

Usually that warning means that cutechess tried to terminate an engine, but the engine process was still alive after the termination timeout. Cutechess should still keep going just fine if that happens. Can you check with Task Manager if there are zombie engine processes still around after you see this warning? And is there some specific free engine with which I could reproduce this warning?

Played against Arasan 19.1 and Bobcat 7.6 with the following batch commands. There are no zombies.

Code: Select all

set C=6
set fname=%~n0

set TCV=60+0.1

set E1="Deuterium-2016"
set E2="Arasan-19.1"
set E3="Bobcat-7.6"

set startpgn=.\pos\bigtestset.pgn
set save=.\games\g-%E1%-%fname%.pgn

cutechess-cli ^
-tournament gauntlet ^
-rounds 25 ^
-games 4 ^
-repeat ^
-draw movenumber=120 movecount=10 score=1 ^
-resign movecount=5 score=700 ^
-openings file=%startpgn% order=sequential start=31000 format=pgn ^
-concurrency %C% ^
-pgnout %save% ^
-wait 5 ^
-recover ^
-engine conf=%E1% tc=%TCV% ^
-engine conf=%E2% tc=%TCV% ^
-engine conf=%E3% tc=%TCV%

[ilari]

Played against Arasan 19.1 and Bobcat 7.6 with the following batch commands. There are no zombies.

Thanks! If there are no zombies then it probably means that terminating the engines took longer than cutechess-cli had patience to wait for.

[Dan Cooper]

First, thanks for this tournament manager.

Now a couple of bug reports:
1) At very fast tc, pretty much anything faster than 60 seconds, cutechess seems to instantiate too many concurrent games. I usually run with concurrency = 8, and after a few rounds I'll notice my CPU spike to 99% and there are 10+ games running. Then it doesn't start new games after the count gets under 8. It waits until all current games finish before starting another 8. I don't notice this type of behavior when running longer tc, such as 3M+2. I constantly see 8 instances running until there are less than 8 games remaining in the tournament.

2) Engines that use 6-man Nalimov take a long time to initialize and cutechess terminates the game before it starts. I've seen this behavior with Crafty, ExChess and Protector. All of my egtb are on SSD.

I have a couple of feature requests also:
1) Can the draw adjudication rules offer an option like TCEC IE pawn pushes and captures resets the adjudication counter?
2) Can support for TLCV broadcasting be added? Ideally, it would generate a uniquely named text file based on the concurrency setting.
3) Support for interrupting and restarting tournaments.

[Ozymandias]

Engines that use 6-man Nalimov take a long time to initialise and cutechess terminates the game before it starts.

6 men Nalimov aren't a good idea for anything else other than analysis.

[ilari]

First, thanks for this tournament manager.

Now a couple of bug reports:
1) At very fast tc, pretty much anything faster than 60 seconds, cutechess seems to instantiate too many concurrent games. I usually run with concurrency = 8, and after a few rounds I'll notice my CPU spike to 99% and there are 10+ games running. Then it doesn't start new games after the count gets under 8. It waits until all current games finish before starting another 8. I don't notice this type of behavior when running longer tc, such as 3M+2. I constantly see 8 instances running until there are less than 8 games remaining in the tournament.

Thanks for the report. Which OS do you use? And have you been able to reproduce this bug with concurrency values below 8? And how long (how many games) do you usually have to wait before you see this bug? I should be able to find out from analyzing the code how this could happen but it would help if the bug was easily reproducible.

2) Engines that use 6-man Nalimov take a long time to initialize and cutechess terminates the game before it starts. I've seen this behavior with Crafty, ExChess and Protector. All of my egtb are on SSD.

The timeout for starting the chess protocol is too low then. I'll increase it or at least make it configurable via command-line. How long does it usually take to fully load one of these engines?

I have a couple of feature requests also:
1) Can the draw adjudication rules offer an option like TCEC IE pawn pushes and captures resets the adjudication counter?

TCEC is using their own fork of cutechess-cli with the adjudication rule you mentioned. I can add that to the next non-patch release.

2) Can support for TLCV broadcasting be added? Ideally, it would generate a uniquely named text file based on the concurrency setting.

We've found all current broadcasting options to be very inadequate so we're actually writing our own implementation for both client and server-side. It's going to take a while to finish though.

3) Support for interrupting and restarting tournaments.

That's also planned. The TCEC fork already has that but it's not a very clean implementation.

[ilari]

1) At very fast tc, pretty much anything faster than 60 seconds, cutechess seems to instantiate too many concurrent games. I usually run with concurrency = 8, and after a few rounds I'll notice my CPU spike to 99% and there are 10+ games running. Then it doesn't start new games after the count gets under 8. It waits until all current games finish before starting another 8. I don't notice this type of behavior when running longer tc, such as 3M+2. I constantly see 8 instances running until there are less than 8 games remaining in the tournament.

Thanks for the report. Which OS do you use? And have you been able to reproduce this bug with concurrency values below 8? And how long (how many games) do you usually have to wait before you see this bug? I should be able to find out from analyzing the code how this could happen but it would help if the bug was easily reproducible.

I opened an issue about this at Github: https://github.com/cutechess/cutechess/issues/91
Unfortunately I could not reproduce it and could not figure out from the code how cutechess-cli could run more than the allowed number of concurrent games. Is it possible that the CPU spike is caused by engines refusing to terminate and continuing to use CPU when they're no longer needed? And it would be great if I could see console output (-debug flag is not needed) of one of these runs. From there it would be easy to see how many games are running at any given point in time.