cutechess-cli in python

[Daniel Shawul]

I got one...

Code: Select all

        import ctypes
        pittichess-noncli = cdll.LoadLibrary("cutechesslib.so")

Seriously though, once you write your backend library in C/C++, then writing a frontend for many languages (scripting and otherwise) is a piece of cake. For example, we use this approach to ease programming the vast array of many-core architecture.

[mcostalba]

I don't understand why all these negative comments for this new nice project.

If Lucas wants to write an engine manager what's the problem?

If he wants to write it in Phyton what's the problem?

I am not good at Python but I really like it, I have developed some small stuff for fishtest testing framework and I found it extremely pleasant to work with. The only serious issue (in part due to my reduced experience) is scalability with program size. When sources get complex you really start to miss the strict type checking and any addition or change can break the code in difficult to find ways (I think this is the reason big Python projects have an almost as big part of code devoted to test suites, this is a workaround to missing type check).

I encourage Lucas to not give up: starting a project is the easy part

[mar]

Have you measured it ? Have you even tried it ? (+same questions on PyPy).

I have measured performance of several interpreted scripting languages. Python really didn't shine. Pypy actually impressed me, even if not competitive with LuaJIT (which is absolutely fantastic).

I claim that you won't see a difference in any realistic testing scenario, because almost all the time is spend waiting for I/O operations.

That remains to be seen, what about GC? Are you sure it won't trigger often during the test run? Are you sure it won't produce additional noise?

Talk is cheap, show us your code...

Didn't like the feedback? Quoting Linus instead of fixing a bug is cheap.
You know very well the hack doesn't prevent the race.

I don't see any point in trying to mimic something that works very well, that was your choice. I have better things to do.

[mar]

Hey, I'm proud of that "nasty" hack!

Well, you shouldn't be
I see a problem with the API where it throws an exception rather than returning null if queue is empty.
However, since you're in a try-except block anyway, why not jobQueue.get(False) and handle the Queue.Empty exception?
If I'm not mistaken you fill the queue with all games to be played in advance.

[mar]

You know very well the hack doesn't prevent the race.

Correction: now I see that the hack actually works in all cases so no race/no hanging is possible

[AndrewGrant]

To restate. I'm proud of my nasty hack.

[mar]

To restate. I'm proud of my nasty hack.

Good for you. And good to know who defecated Lucas' code

[AndrewGrant]

I have measured performance of several interpreted scripting languages. Python really didn't shine. Pypy actually impressed me, even if not competitive with LuaJIT (which is absolutely fantastic).

You understand that python is not actually managing the running engines, right? Once the engines are launched and given commands they don't give a damn about the python code that spawned them. The only control that python has over them is knowing their process IDs and being able to kill them when python wants to.

So for performance are you worried about the time between sending and receiving commands? Are you suggesting we should be worried about the millisecond it takes to give the engine the command, get the move, and update the board? Are you afraid of wasting a few milliseconds a game? If the answer is yes, you better launch a cutechess-x86-assembly project of your own.

That remains to be seen, what about GC? Are you sure it won't trigger often during the test run? Are you sure it won't produce additional noise?

Yes, and Yes.

My tests showed an average of less than 1 garbage collection cycle per game, taking, on average, less than 1 millisecond, at most 3 milliseconds.

In thousands of games across multiple cores, a garbage collection cycle was never run in such a way that it affected the elapsed Time calculation for an engine. Which, is the expected result, because despite the mysticism behind garbage collection, it is not entirely unpredictable.

Didn't like the feedback? Quoting Linus instead of fixing a bug is cheap.
You know very well the hack doesn't prevent the race.

Correction: now I see that the hack actually works in all cases so no race/no hanging is possible

After you took some time to think about it you came around to see that it worked. If you don't like the response Lucas gave, then you SHOULD write some code to show us. I tried a dozen things, and found this is be the most simple, reliable solution I could come up with. If you know of a better method, please share. I am always open to learning new things.

[AndrewGrant]

the queue.get(false) with exception handling is nice, but what if we want to fill the queue with games for multiple pairings? We have engines A, B, and C. We want A to play 25 vs B, B to play 25vs C, and C to play 25 vs A. We let each pairing play out their 25 games, before closing down those Engine pairings. There is more to it than you looked for.

[lucasart]

what about GC? Are you sure it won't trigger often during the test run? Are you sure it won't produce additional noise?

There's no GC in CPython, which uses reference counting. This is essentially equivalent to C++ RAII, where the object are destroyed when no longer needed (goes out of scope in C++, reference count drops to zero in CPython).

It's actually PyPy that uses GC. So I am surprised that you are now using GC as a last attempt argument to defend your position, because you said earlier that PyPy was faster than CPython (with of course no evidence to support this claim).

I recommend you learn a bit more about Python before crapping on it.