Page 1 of 4

cutechess-cli in python

Posted: Thu Jun 16, 2016 2:59 am
by lucasart
Code speaks louder than words, so here it is:
github.com/lucasart/zinc

Please note that:
* I do not intend to replicate all the functionalities of cutechess-cli. Only the ones useful for engine testing.
* This script is intended for programmers, not end-users, so I will not write a CLI (let alone a GUI), with idiot-proof input validation code. I will write some documentation, and try to keep the code clean and self-documentating, of course.

Feedback, suggestions, bug reports, bug fixes, are welcome. So are pull requests for new functionalities, provided they don't deviate from the aforementioned project goal of simplicity and minimalism.

Enjoy!

PS: Huge credit goes to Niklas Fiekas, for his python-chess library, without which this project would not exist!

Re: cutechess-cli in python

Posted: Wed Jun 22, 2016 11:49 am
by lucasart
Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:
  • CLI: not implemented. just configure the parameters at the top of the script and run it.
  • JSON: not implemented. not needed for now (no CLI).
  • Variants: not implemented. will only support Chess960.
  • Gaviota TB: not implemented. may add support for Syzygy.
  • Tournaments: not implement. Only 2 engine matches for now.
  • SPRT: not implemented. will implement.
  • ratinginterval: not implemented. will implement.
  • openings: only EPD, sequentially (repeated).
  • pgnout: not implemented.
  • recover: not implemented.
  • repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
  • restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
  • proto: only UCI. won't support CECP.
  • timemargin: not implemented.
  • book: not implemented.
  • ponder: not implemented.
All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs :lol:

Re: cutechess-cli in python

Posted: Wed Jun 22, 2016 12:20 pm
by hgm
So apart from almost everything, it behaves exactly the same. :wink:

Re: cutechess-cli in python

Posted: Wed Jun 22, 2016 12:46 pm
by lucasart
hgm wrote:So apart from almost everything, it behaves exactly the same. :wink:
Trolling as usual. You did not look at it, nor even looked at what is available in cutechess-cli.

It has all the features of cutechess-cli that are useful for fishtest, and more.

Re: cutechess-cli in python

Posted: Wed Jun 22, 2016 2:17 pm
by hgm
I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'. :lol:

Re: cutechess-cli in python

Posted: Thu Jun 23, 2016 2:00 pm
by lucasart
lucasart wrote:Some good progress was made on the project, as you can see on github: https://github.com/lucasart/zinc.git. Kudos to Andy for his great contributions.

Here's the gap-analysis compared to cutechess-cli:
  • CLI: not implemented. just configure the parameters at the top of the script and run it.
  • JSON: not implemented. not needed for now (no CLI).
  • Variants: not implemented. will only support Chess960.
  • Gaviota TB: not implemented. may add support for Syzygy.
  • Tournaments: not implement. Only 2 engine matches for now.
  • SPRT: not implemented. will implement.
  • ratinginterval: not implemented. will implement.
  • openings: only EPD, sequentially (repeated).
  • pgnout: not implemented.
  • recover: not implemented.
  • repeat: the only behaviour available. EPD positions are always played twice, with colors reversed.
  • restart: engines are never restarted. don't see any reason to restart engines, other than slowing things down, so won't support it.
  • proto: only UCI. won't support CECP.
  • timemargin: not implemented.
  • book: not implemented.
  • ponder: not implemented.
All the rest is available, and behaves the same was as in cutechess-cli, modulo bugs :lol:
Added Chess960 and PolyGlot opening book (weighted random selection up to book depth).

Re: cutechess-cli in python

Posted: Thu Jun 23, 2016 2:13 pm
by lucasart
hgm wrote:I looked at the list you posted, and almost everything that makes cutichess-cli a useful tool seems to be amongst the 'not supported'...

So the title of the thread is a bit misleading. This isn't like cutichess any more than any tool to play simple matches between UCI engines. As it is based on Python rather than Qt, 'pittichess-noncli' would be a more apt name.

It is a bit like saying that micro-Max is 'Stockfish in plain C'. :lol:
It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.

In my experience -- and I've used cutechess-cli a lot -- the CLI part of cutechess-cli adds zero value, in terms of usability. It's only necessary because C++ is a compiled language. There are so many parameters you need to put in cutechess-cli, that you end up having to write a shell script (and a JSON) around it, edit that script, and run it. That's exactly what you do with zinc: you edit the config at the top of the Python script and run it.

Re: cutechess-cli in python

Posted: Thu Jun 23, 2016 3:09 pm
by mar
lucasart wrote:It's going to replicate all the useful functionalities of cutechess-cli. Of course, it's not finished, but already you can see that development is several orders of magnitude faster than cutechess-cli; precisely because it's in Python, and I don't have to deal with all the bureaucracy of C++ / Qt to get the job done.
:lol: well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.
Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.

So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?
Also you don't need to depend on Qt at all to write something like cutechess-cli.

PS this nasty hack should prevent races? :D C'mon, you can do better ;)

Code: Select all

            # HACK: We can't just test jobQueue.empty(), then run jobQueue.get(). Between both
            # operations, another process could steal a job from the queue. That's why we insert
            # some padding 'None' values at the end of the queue
            job = jobQueue.get()
            if job == None:
                return

Re: cutechess-cli in python

Posted: Thu Jun 23, 2016 5:47 pm
by AndrewGrant
Hey, I'm proud of that "nasty" hack! :)

Re: cutechess-cli in python

Posted: Thu Jun 23, 2016 10:46 pm
by lucasart
mar wrote: :lol: well, Python has the slowest interpreter I've ever seen. This is not about performance, sure (or is it? Pypy for the win!)
Plus you depend on python itself, a piece of bloat that has to load first to run your little script.
Have you measured it ? Have you even tried it ? (+same questions on PyPy).
I claim that you won't see a difference in any realistic testing scenario, because almost all the time is spend waiting for I/O operations.
Why not Lua or something else that's lightweight (which you apparently like)? I guess because Python is cool.
I don't know anything about Lua, so I can't comment on that. And yes, Python is cool.
So you've glued together some libs and made something that runs, congrats. How does it relate to the alleged "bureucracy" of C++ is beyond me. Have you just discovered what scripting languages are good for?
I never said it was hard. On the contrary, I'm saying it's easy, thanks to Python itself, and to the python-chess library of Niklas.
Also you don't need to depend on Qt at all to write something like cutechess-cli.
Agreed.
PS this nasty hack should prevent races? :D C'mon, you can do better ;)
Talk is cheap, show us your code...