I had to redo my CLOP experience, because there was a severe flaw: the knight value was used to make search decisions (eval pruning margin was a function of the kight value and the depth). An easy trap to fall into! For example if it appears that reducing the eval margin increases elo, then CLOP would penalise the knight value because of that, which isn't what we want. So it's very important to look at side effects in the program of the values to optimize.
Also I arbitrarly imposed knight=bishop to reduce the dimension of the problem to 4 i/o 5. Once I get the optimal values of the other pieces, I'll optimize in dimension 1 by imposing knight=x-delta bishop=x+delta, where x is the knight=bishop value previously determined.
And after that I'll tackle the search pruning techniques, and optimize their margins.
lucasart wrote:Also I arbitrarly imposed knight=bishop to reduce the dimension of the problem to 4 i/o 5. Once I get the optimal values of the other pieces, I'll optimize in dimension 1 by imposing knight=x-delta bishop=x+delta, where x is the knight=bishop value previously determined.
Probably not a very good idea. CLOP should be able to optimize everything at the same time more efficiently.
Rémi Coulom wrote:"Mean" is the weighted mean of all samples. I recommend it as a choice of weight values
Would you consider adding another column to show the weighted stdev of parameters. After thousands of samples, I would be curious to see how much CLOP still moves the paremeters around. Or even LCB/UCB 95% (weighted) if that is feasible
Added to the TODO. But I have a lot of plans with much higher priority, so the next release of CLOP may take a lot of time.
Name CLOPUCI
Script ./cutechess-cli/clop-cutechess-cli.py
IntegerParameter P 0 150
IntegerParameter N 100 700
IntegerParameter B 100 700
IntegerParameter R 200 1000
IntegerParameter Q 300 2000
Processor cpu1
Processor cpu2
Replications 2
DrawElo 100
H 3
Correlations all
P is the opening value of a pawn (endgame value is 100, it's the reference point for everything). The rest is obvious
Will I ever see a QLR max inside the bounds like that ? OK I know, I'm impatient and I need to wait thousands of games, but I just want to be sure I won't wait for nothing :p
This is taking forever... I'm thinking I could run the same CLOP experiment on my other computer at the same time and merge them at the end.
If I use the same clop file and the same everything else (connection script, engines), can I just concatenate the .dat files and merge the experiments ? I would imagine the .log file isn't important, right?
lucasart wrote:This is taking forever... I'm thinking I could run the same CLOP experiment on my other computer at the same time and merge them at the end.
If I use the same clop file and the same everything else (connection script, engines), can I just concatenate the .dat files and merge the experiments ? I would imagine the .log file isn't important, right?
The log file is not important.
Merging .dat files would be doable, but it cannot be done by just concatenating the files. You'll have to look at the files to understand its format (lines starting with "S" is start, lines starting with "R" is result, and each game is identified by its seed). You cannot have more than one game with the same seed, so you'd have to re-label the seeds in order to merge the data files.
If your machines are connected in a network, the right approach is to have one instance of CLOP running on one machine, and let the connection script distribute the jobs.
