A tuning idea for engines

[MikeG]

Suppose we wish to add ELO to an engine by decreasing the BF through use of aggressive LMR or pruning . Since aggressive pruning only begins to fails due to errors in the evaluation, I wonder if there might be some benefit (or maybe some has already tried this) to proceed as follows:

0) Take some non-perfect engine at starting point “0”. The goal will be to add ELO through more aggressive LMR’s and/or pruning. This in turn will be made possible by improving the eval. But before we mess with the eval, we implement step 1:

1) Step 1 will be to increase aggressiveness of pruning significantly (most likely well beyond the current optimal behavior). This should cause a (hopefully temporary) drop in ELO.

2) Step 2 would be to tune the evaluation of the aggressive version , until you can reach some improvement.

3) Re-tune the LMR / pruning to find maximal ELO (ideally, would be like to end of with something more aggressive than version 0).

The basic thought is, perhaps this aggressive pruning version in stage 1 would be an great “test bed” bed” – although it would have lower ELO than stage 0, it might be more sensitive to a tweak which happens to reduce the error (as compared to similar evaluation tweaks in the base version). It would be something like a highly selective evolutionary pressure on the eval function, as compared to base case, where by design, less aggressive pruning must be done to cover up the evaluation errors involved in ranking moves.
Has anyone tried something like this?

[Michael Sherwin]

RomiChess is the perfect candidate! The eval can be improved drastically by someone that knows what they are doing! You can find the sources at the WBEC website.

[stegemma]

I think that you could get the same (and maybe better) results by using a "full pruning". I mean that if you prune all the moves but the best one selected by evaluation function, you'll get a worst program that rely ony on evaluation. In this way, you can work only on evaluation and only at the end (when you can't get more from evaluation) you'll turn back to standard pruning. The interesting thing is that in that way you can play a lot of games, for any change in evaluation, because the BF becomes exactly 1. Or maybe what you get is just to test pseudo-random games....

[MikeG]

Interesting thought, but I imagine that would be going to far. I think testing would be a challenge. Pragmatically, I think after a tweak you would want to run the engine through a gauntlet and have a score to measure.

A full-pruning version like you suggest might be well suited to an genetic approach to eval tuning, because you could run millions of games (per second)? with many different evaluation components.

My original idea was less complicated. I was based on the fact that when tuning multi-variable systems, it is often necessary to de-tune one variable to get away (perhaps temporarily) from a "local maxima". And maybe this concept could be applied to the interaction between eval and pruning parameters.

It will likely be December before I have enough time to try any such project myself. I thought maybe others here had tried similar stuff, or might find it worth s try.

[michiguel]

Interesting thought, but I imagine that would be going to far. I think testing would be a challenge. Pragmatically, I think after a tweak you would want to run the engine through a gauntlet and have a score to measure.

A full-pruning version like you suggest might be well suited to an genetic approach to eval tuning, because you could run millions of games (per second)? with many different evaluation components.

My original idea was less complicated. I was based on the fact that when tuning multi-variable systems, it is often necessary to de-tune one variable to get away (perhaps temporarily) from a "local maxima". And maybe this concept could be applied to the interaction between eval and pruning parameters.

Do you think that eval and pruning parameters are correlated (with the obvious exceptions of futility margins etc.)?

Miguel

It will likely be December before I have enough time to try any such project myself. I thought maybe others here had tried similar stuff, or might find it worth s try.

[MikeG]

I would suppose there is correlation in the sense that higher uncertainties in the eval will limit the extent of pruning possible. But please elaborate if you have some other thought (?)