Daniel Shawul wrote:Ok that makes sense, we fake CLOP to think it has 128 workers and thus extract 128 different parameters from it. I am not ready to write such code yet . Anyway I find the connection scheme that CLOP uses that invokes a script for each game rather inconvenient. Even on a single machine, I had to disable time-taking stuff such as EGBB loading, and use a faster script etc. Luckily cutechess-cli itself is fast enough. I would prefer if a script can be re-usable to avoid this cost with a pipe connection or something like that just like winboard. But I understand as a black-box optimizer, it is convenient for CLOP to have it the way it is but not so much for the programer.
I wrote this for my Go program, where games have to be slow.
It should be really easy to modify clop such that it passes an additional parameters to the script (the number of replications) and collects all the results with just one script invocation. Much easier than setting up a client/server infrastructure with queues on your cluster.
I see, a monte-carlo searcher may indeed take a long time to finish a game. I am using fixed depth searches for tuning eval with depth 3,5 and max 8. The first one takes less than a second to complete a game, and depth=8 usually less than 5 secs. I use the latter for tuning some terms such as king safety.
Though the Replications parameter allows to play many games with same set of parameters, it still invokes the script specified number of times, so a job queue may still be required. I think that it will be slighlty inefficient interms of optimization, but that is acceptable compared to its advantage to lower latency. It will establish the elo difference between two players to a higher certainty before adjusting the parameters.
Btw I am having some problems with clop after 30000 games or so in which the database is lost. If the engine or cutechess-cli hangs, the .dat file is lost with them. The cutechess-cli process is seen as a zombie but when I stop that and restart the data will be gone. This happened to me twice when I run it overnight. I am using CLOP 0.0.9
. Anyway I find the connection scheme that CLOP uses that invokes a script for each game rather inconvenient. Even on a single machine, I had to disable time-taking stuff such as EGBB loading, and use a faster script etc. Luckily cutechess-cli itself is fast enough. I would prefer if a script can be re-usable to avoid this cost with a pipe connection or something like that just like winboard. But I understand as a black-box optimizer, it is convenient for CLOP to have it the way it is but not so much for the programer.