To Ilari : CLOP + cutechess-cli

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: To Ilari : CLOP + cutechess-cli

Post by ilari »

Rémi Coulom wrote:
ilari wrote:- When the game or games have been played Cutechess-cli should just print "W", "L" or "D".
Beware that CLOP expects only one game per script invocation.

Rémi
Yes, I figured that out after playing with DummyExperiment for a while. Thanks for CLOP by the way, what a cool tool.

The problem is that cutechess does not support a list of engines but just 2, so to make seed >> 1 work with cutechess you have somehow to tell cutechess about the engines list.
It's not really a problem. It would be easy to modify Cutechess-cli to accept a list of engines as a -clop parameter. Then it would just pick one based on the seed and replications.

BTW, I already made the changes to tune parameters against one opponent, and I'm now running my first test optimizing Sloppy's static evaluation weights against Crafty. It's working - the win percentage is going up and the weights are starting to look somewhat sensible after only 300 games.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: To Ilari : CLOP + cutechess-cli

Post by mcostalba »

ilari wrote: But I'm still pondering whether or not I should leave that stuff for a script that operates between CLOP and Cutechess-cli.
To have the complete picture, consider that CLOP is designed to distribute work to a cluster of machines, the id parameter is the name of the node, so perhaps the correct approach to avoid future shortcomings is to foreseen a dispatcher between CLOP and cutechess....anyhow you are the developer ;-)
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: To Ilari : CLOP + cutechess-cli

Post by ilari »

Okay, I decided to go with the script approach. It was just the better and easier option in many ways. Here: http://koti.mbnet.fi/~ilaripih/bin/clop ... ess-cli.py

It's in Python, so those running Windows may have to install the Python implementation from www.python.org first.

The script has to be modified a bit before it can be used. The relevant parts are at the beginning of the script:

Code: Select all

# Path to the cutechess-cli executable.
# On Windows this should point to cutechess-cli.exe
cutechess_cli_path = '$HOME/chess/cutechess-cli/cutechess-cli.sh'

# The engine whose parameters will be optimized
engine = 'conf=Sloppy-0.2.2'

# Format for the commands that are sent to the engine to
# set the parameter values. When the command is sent,
# {name} will be replaced with the parameter name and {value}
# with the parameter value.
engine_param_cmd = 'setvalue {name} {value}'

# A pool of opponents for the engine. The opponent will be
# chosen based on the seed sent by CLOP.
opponents = [
    'conf=Fruit-2.1',
    'conf=Crafty-22.10',
    'conf=Glaurung-2.2',
    'conf=Gaviota-0.8.0'
]

# Additional cutechess-cli options, eg. time control and opening book
options = '-both tc=40/1+0.05 book=$HOME/chess/pg_books/performance.bin -draw 80 1 -resign 5 500'
* <cutechess_cli_path> should be the full path to the cutechess-cli executable or launcher script.
* <engine> can be anything that you would normally put in '-fcp', so you can also use 'cmd=engine.exe dir=path_to_engine proto=xboard', etc.
* <engine_param_cmd> is the command that is sent to your engine, and you can change the syntax freely, as long as the {name} and {value} fields are there. To optimize UCI options you can set engine_option_cmd to 'setoption name {name} value {value}'. The option
* <opponents> is a list of opponents to play against. There has to be at least one opponent.
* <options> should have all the rest of cutechess-cli's parameters.

This script can be used with any Cutechess-cli version, so go ahead and give it a try.
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: To Ilari : CLOP + cutechess-cli

Post by ilari »

ilari wrote:This script can be used with any Cutechess-cli version, so go ahead and give it a try.
Turns out that I was wrong - no released version of cutechess-cli can take negative parameter values. This bug is fixed in the GIT repository: http://repo.or.cz/w/sloppygui.git
So those who compile cutechess-cli from source can use this script. Others should wait until I release a new version. Sorry 'bout that.

Also, I forgot to mention that <engine_param_cmd> will be sent as an initialization string to the engine, that is, before even starting the xboard/uci protocol.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: To Ilari : CLOP + cutechess-cli

Post by mcostalba »

ilari wrote:Okay, I decided to go with the script approach. It was just the better and easier option in many ways. Here: http://koti.mbnet.fi/~ilaripih/bin/clop ... ess-cli.py
Thanks Ilari ! I will test this week end :-)
Michel
Posts: 2272
Joined: Mon Sep 29, 2008 1:50 am

Re: To Ilari : CLOP + cutechess-cli

Post by Michel »

no released version of cutechess-cli can take negative parameter values.
Are you absolutely sure this is the case? I use negative parameter values all the time and I checked in the debug file that they are sent to the engine.

One has to quote the negative values.
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: To Ilari : CLOP + cutechess-cli

Post by ilari »

Michel wrote:
no released version of cutechess-cli can take negative parameter values.
Are you absolutely sure this is the case? I use negative parameter values all the time and I checked in the debug file that they are sent to the engine.

One has to quote the negative values.
Oh you're right, I must have been too tired when I wrote my previous message. And the parameters will be inside quotes anyway when the script sends them to cutechess-cli. So yeah, no need to wait for a new version after all. :D
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: To Ilari : CLOP + cutechess-cli

Post by mcostalba »

After 4 hours of hardcore hacking I was able to setup the thing on Windows 7 with Qt SDK mingw compiler.

Regarding your script I needed to change the following:

Code: Select all

sts = os.waitpid&#40;process.pid, 0&#41;
because on Windows there is not support for pid:

http://stackoverflow.com/questions/6647 ... -processes

So I workaround with:

Code: Select all

process.wait&#40;)
But the most amazing thing has been to make CLOP to run. In the README it is written:

Code: Select all

Windows & Mac
-------------
Get the Qt SDK. Project files in programs/clop/compqt/* can be opened in Qt
Creator and compiled.
I am wondering if the author as ever tried to do as he advertised...

Just for starter the QT SDK with mingw does not compiles CLOP due to the -ansi option in the QMAKE_CXXFLAGS_WARN_ON qmake variable (you need to remove -ansi from the .pro files), then I was unable to make the windows version of CLOP to launch the python glue script DummyScript.py, perhaps is because I missed something, anyhow after a good time spent googling+hacking around I workaround the issue converting the script in an exe file thanks to:

http://www.py2exe.org/

BTW also this was not immediate due to the fact that I downloaded the 64 bit version of py2exe (I have Windows 7 64 bit), instead I needed the 32 bits version.

Finally, after a lot of work, I have now my PC running the thing !

I have written this notes with the aim to avoid the pain to someone else.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: To Ilari : CLOP + cutechess-cli

Post by Daniel Shawul »

sts = os.waitpid(process.pid, 0)
Gee.I spent hours but I couldn't make it through apparently because of the bug you discovered.
I still have problems with cutechess-cli .json file which seems to ignore inisstring. Some uci engines take all the processors unless told to use threads=1.
Just for starter the QT SDK with mingw does not compiles CLOP due to the -ansi option in the QMAKE_CXXFLAGS_WARN_ON qmake variable (you need to remove -ansi from the .pro files), then I was unable to make the windows version of CLOP to launch the python glue script DummyScript.py, perhaps is because I missed something, anyhow after a good time spent googling+hacking around I workaround the issue converting the script in an exe file thanks to:

http://www.py2exe.org/
I used the older version QLR for windows (already compiled) and it seems to work both ways. Calling the script direcly or after convering to py2exe.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: To Ilari : CLOP + cutechess-cli

Post by mcostalba »

Daniel Shawul wrote: I still have problems with cutechess-cli .json file which seems to ignore inisstring. Some uci engines take all the processors unless told to use threads=1.
You may want to set:

Code: Select all

engine_param_cmd = 'setoption &#123;name&#125; value &#123;value&#125;'
in clop-cutechess.py instead of the shipped

Code: Select all

engine_param_cmd = 'setvalue &#123;name&#125; &#123;value&#125;'
The former seems the correct one for UCI engines, I don't know when is used the second one....