Tuning parameters.

[Robert Pope]

But you can absolutely do it one parameter at a time if you want. The plus side is that it's very simple to implement and you don't have to deal with the math or coding complexity of gradient descent. Tweak a parameter and see if it reduces the error, then try tweaking another one. It may not be efficient, but it works, and the wasted time can be done while you are asleep.

[Viz]

Side note - instead of 64 squares in PSQT you probably would want to have 32 mirrored.
At least for everything that is not pawns.

[hgm]

But you can absolutely do it one parameter at a time if you want. The plus side is that it's very simple to implement and you don't have to deal with the math or coding complexity of gradient descent. Tweak a parameter and see if it reduces the error, then try tweaking another one. It may not be efficient, but it works, and the wasted time can be done while you are asleep.

A reasonably simple, yet efficient method that worked for me is to change all parameters simultaneously, by a combination of a random change and a repetition of steps that were successful in the past. The latter is implemented as the moving average of recent successful steps. If a change is succesful you add that change to a 'bias vector', if not, you multiply the bias vector by, say, 0.9. The changes to be tried will be this bias vector plus a random component.

You could also make the magnitude of the random change adapt to the success rate. E.g. by multiplying it with a factor after a successful step, and dividing it by a smaller factor otherwise. This would decrease the size of the changes when the success rate gets low because you are already so close to the optimum that large changes make you jump away from it.

[Viz]

From a tuning large array of parameters standpoint there are literally 2 good methods - texel tuning and SPSA.
Have never used 1st myself but SPSA works surprisingly well and converges surprisingly fast for this things.

[hgm]

I would expect SPSA to struggle in case of a 'stiff' object function, such as 1e4*(SUM x_k^2) - 10(SUM x_k)^2 + 1e-4*(x_1 - 100)^2, with 1000 parameters x_k. (The fist two terms define a channel with bottom 0 when all x_k are equal, but increases steeply when you get away from that path.)