How Stable is CuteChess-Cli (or what am I doing wrong)?

[Steve Maughan]

I had assumed CuteChess-Cli must be rock solid as it seems to be used by all serious (and even semi-serious) chess programmers.

So when playing games between development versions of Maverick I was always irked to see messages such as "pipe terminated". And often at fast time controls CuteChess-cli would freeze. I assumed there must be a bug in Maverick's UCI interface code.

I've trawled though it backwards and forward, up and down, right and left. I cannot find a problem.

Then I decided to stress test other engines. I chose TJChess and Fruit 1.0 - both seem to be stable.

I then renamed the executables "Maverick", replaced my exes and ran the same script. CuteChess-Cli still crashed after a number of games. This make me think it's not as stable as I first thought.

I'd like to do 5-ply fixed depth games. Here's the script I'm using and I'm running on a 4 core i7 (+HT).

Where's the problem?

Does anyone else run fixed depth games?

I'm using a version of CuteChess-Cli dated July 12th 2013.

Thanks,

Steve

Code: Select all

@ECHO OFF
 
SET varYYYY=%DATE:~10,4%
SET varMM=%DATE:~4,2%
SET varDD=%DATE:~7,2%
SET varTodaysDate=%varYYYY%%varMM%%varDD%

SET PGN_DATABASE=Tournament%varTodaysDate%.pgn

c:\CuteChess\cutechess-cli -engine name="Maverick M0" cmd=Maverick dir=c:\CuteChess\MaverickBase -engine name="Maverick M1" cmd=Maverick dir=c:\CuteChess\MaverickNew1 -engine name="Maverick M2" cmd=Maverick dir=c:\CuteChess\MaverickNew2 -engine name="Maverick M3" cmd=Maverick dir=c:\CuteChess\MaverickNew3 -engine name="Maverick M4" cmd=Maverick dir=c:\CuteChess\MaverickNew4 -each option.Hash=128 timemargin=5 proto=uci depth=5 tc=40/30 book=c:\CuteChess\obook.bin -pgnout c:\CuteChess\Games\%PGN_DATABASE% -games 2 -rounds 200 -concurrency 4 -wait 5 

pause 10

[gladius]

It has been rock solid for the SF testing framework. Here is the an example of the command we use:

cutechess-cli -repeat -rounds 1000 -tournament gauntlet -pgnout results.pgn -resign movecount=3 score=400 -draw movenumber=34 movecount=8 score=20 -concurrency 3 -openings file=8moves_GM.pgn format=pgn order=random plies=16 -engine name=stockfish cmd=stockfish option.Hash=128 option.OwnBook=false -engine name=base cmd=base option.Hash=128 option.OwnBook=false -each proto=uci option.Threads=1 tc=60+0.05

You could try the version Ilari compiled for SF, it's at https://github.com/mcostalba/FishCookin ... li-win.zip

[Steve Maughan]

Thanks Gary - Maybe the depth=5 command in my script is a problem. I notice your time control us tc=60+0.05. Is this "Game in 60 secs plus 0.05 secs per move"? I didn't know CuteChess could do this type of time control.

Thanks again,

Steve

[Ajedrecista]

Hello Steve:

Thanks Gary - Maybe the depth=5 command in my script is a problem. I notice your time control us tc=60+0.05. Is this "Game in 60 secs plus 0.05 secs per move"? I didn't know CuteChess could do this type of time control.

Thanks again,

Steve

I think you are right: 60+0.05 means 'game in 60 seconds plus 0.05 seconds per move' (a Fischer TC).

Regarding the question of fixed depth, I ran a fixed depth experiment a year ago:

Diminishing returns in fixed depth testing revisited.

You can see there which command line I used. Just for the record, the version of cutechess.-cli was 0.5.1 for Windows (32-bit), last modified on 6th January, 2012. I want to thank Adam Hair again for his help teaching me how use cutechess-cli. I ran some tenths of thousands of games without crashes.

I am not sure if I got some messages of 'pipe terminated', it is possible, but anyway there were not a problem for me if they appeared, just as killTimers issue.

I hope you will find this info useful.

Regards from Spain.

Ajedrecista.

[Steve Maughan]

It looks as if I might have found the problem.

It wasn't with CuteChess-Clli. Hence the follow-up post - I wouldn't want people to think CuteChess-cli is buggy, when it's not.

The bug I found (there may be more) was to do with calculating the target move time when the time is very low.

Thanks,

Steve