CLOP for Noisy Black-Box Parameter Optimization

[Rémi Coulom]

More generally, you can try to be creative to reduce the dimensionality of the optimization problem.

Thanks for your quick answer to previous question. Here is the next one

Why don't you reduce the dimensionality by yourself ?

I mean user asks to tune p1,....,p8 and then he sets also a new parameter that is "dimensionality": given dimensionality = 2 you tune the two derived values c1 and c2 that are derived from p1,..p8 for instance (but here you are much more creative than me) by a linear combination of p1...p8.

If you remember the ampli+bias idea that Joona and me reported, this is a kind of generalization of that idea.

What do you think ?


P.S: After many months tuning SF I made up my mind that the secret of a good tuning is the choice of the starting variables to tune. So a mapping of P1,...,Pn to C1,...,Ck and tune Ck could yield, if done properly, a much faster and better tune.

There are ways to do it. I am thinking about using sparse regression techniques to handle the high-dimensional case.

But nothing will completely replace the intelligence of the user. It is like building an evaluation function. You can try to use a universal function approximator, with the chess board as input and the evaluation as output, and optimize that approximator somehow. But no generic solution will work better than manually building domain-specific features.

Rémi

[Rémi Coulom]

Another question. How does a parameter that can't be tuned (let's say it doesn't effect the outcome of the game at all), interfere with other parameters in the same suite?

Would removing that ineffective parameter improve the result or is it disregarded anyway?

Adding a parameter that has no effect should not hurt the performance of CLOP much. It will hurt in terms of CPU time to compute the regression. But it should not hurt much in terms of the quality of the estimated optimal parameters for a given number of games played.

Rémi

[Rémi Coulom]

It seems to me that you would get a similar effect if you let your CLOP parameters represent deltas from your current parameter values, and then project the solution vector onto the subspace corresponding to the k smallest eigenvalues of the hessian.

There is no need to do that at all. As I said, CLOP already does it when it is run with "Correlations all". Any linear transformation of the parameter space should not change the behaviour of CLOP.

Non-linear transformations might be more interesting, though. Any transformation that makes the winning rate a more "quadratic" function of parameters is good. That's why CLOP has the "GammaParameter" option. That's also why I suggested to use p1, p2, p3 instead of p1, p2 * p1, p3 * p1. These are non-linear transformation that might help to make data more quadratic-like.

Rémi

[petero2]

It seems to me that you would get a similar effect if you let your CLOP parameters represent deltas from your current parameter values, and then project the solution vector onto the subspace corresponding to the k smallest eigenvalues of the hessian.

There is no need to do that at all. As I said, CLOP already does it when it is run with "Correlations all". Any linear transformation of the parameter space should not change the behaviour of CLOP.

The eigenvector corresponding to the smallest (most negative) eigenvalue corresponds to the direction in parameter space that has the largest effect on playing strength. The idea was to ignore adjustments in directions that has negligible effect on the playing strength, on the assumption that it is better to stay close to the original values in that case.

Consider the following example, where I tried to tune the four parameters called qV, rV, mV and pV, which represent piece value corrections for queen, rook, bishop/knight and pawn. After about 6000 games, CLOP reported the following parameter estimates and hessian: (using octave syntax)

Code: Select all

#  qV  rV mV pV
V=[-14 23 26 -2]';
H = [
  -1.9209  -2.0485   3.9416   3.8301
  -2.0485  -2.7984   4.5237   1.7491
   3.9416   4.5237  -8.2445  -5.6973
   3.8301   1.7491  -5.6973  -9.4130
];

The eigenvectors and eigenvalues are:

Code: Select all

[v,L]=eig(H)
v =

  -0.340050  -0.025676   0.424797   0.838603
  -0.303696  -0.510986   0.652862  -0.469502
   0.634159   0.457622   0.621699  -0.043763
   0.624481  -0.727197  -0.082519   0.272760

L =

Diagonal Matrix

  -18.134820           0           0           0
           0   -4.463421           0           0
           0           0   -0.044604           0
           0           0           0    0.266044

The projection onto the subspace spanned by the first k eigenvectors is:

Code: Select all

vk=v(:,1:k); Vk=sum(diag(vk'*V)*vk',1)

In this case it looks like the effect of moving in the direction of the last two eigenvectors is questionable because the noise level is too high. I you believe that the original parameter values are reasonable, it seems more logical to not move in those directions.

Setting k=2 gives:

Code: Select all

k=2; vk=v(:,1:k); Vk=sum(diag(vk'*V)*vk',1)
Vk =

  -4.4760  -4.9538   9.1501   6.7027

Although, I admit that I'm not sure if this method actually improves anything, i.e. if it gives usable parameter adjustments with fewer number of played games.

[Rémi Coulom]

Although, I admit that I'm not sure if this method actually improves anything, i.e. if it gives usable parameter adjustments with fewer number of played games.

So, in case the parameters are independent, that would mean reducing the range of values for parameters that don't have a strong influence on strenth. That sounds strange, and I don't expect it would help. I may be wrong, though.

Rémi