interpolation scoring tuning

[bob]

Here's an interesting thing I have run into with the "interpolation" scoring...

I had two terms, rook on 7th for middle-game and endgame. I ran an automated script that first discovered that the MG score needed to be doubled, and then it fiddled with the EG score and found it needed to be doubled. I doubled both and the Elo went _down_. Here's why...

this interpolation approach varies the score from MG to EG as material comes off. So there are two parts to this tuning.

(1) which should be bigger, MG or EG? Debatable here, and not so imporrtant.

(2) what should the min and max be? Slightly different question, but important. What my testing above was saying was "rook on 7th needs to be larger in general, and by doubling either EG or MG score, the overall average score goes up. Whether the EG or MG score should be bigger was not getting tested, I was discovering that the overall score needed to go up.

I eventually doubled one end of the score, then ran a tuning run to tweak both ends proportionally to see if one should be bigger than the other or if equal is actually the best value. However, this was not obvious initially and the results were misinterpreted completely...

[mathmoi]

I had two terms, rook on 7th for middle-game and endgame. I ran an automated script that first discovered that the MG score needed to be doubled, and then it fiddled with the EG score and found it needed to be doubled. I doubled both and the Elo went _down_. Here's why...

How does this automated script discovered that the rook_on_7th score needed to be doubled? Do you use some kind of learning (or another type of heuristic) to find your evaluations params values (a la Larry Kufman)?

[BubbaTough]

Yes this kind of thing is interesting and makes tuning challenging. I see it less of an interpolation thing, and more of an interaction between two variables thing. Another example would be: a) increase rook on open file = +, b) increase rook mobility = +, a & b = - (because overall rook increase in value causes too many sacrifices of two pieces for rook and pawn). When it happens in interpolated variables figuring out the root issue certainly is more challenging though.

This kind of thing happens so much, it is one of the things I think causes so many programs to get stuck in local maxima, where improvement gets difficult.

-Sam

[bob]

I had two terms, rook on 7th for middle-game and endgame. I ran an automated script that first discovered that the MG score needed to be doubled, and then it fiddled with the EG score and found it needed to be doubled. I doubled both and the Elo went _down_. Here's why...

How does this automated script discovered that the rook_on_7th score needed to be doubled? Do you use some kind of learning (or another type of heuristic) to find your evaluations params values (a la Larry Kufman)?

I am doing this "manually". I just say "play a match, then adjust this particular score by some percentage (up or down), then play another match". So I pick the term to tweak, and how it is going to be tweaked. And let the script run N matches. I then look at the Bayeselo output which has each tweaked version in one large combined pgn file, to see which value is best. Sometimes I then refine the test if I used a "coarse" resolution on the adjustment...

[bob]

Yes this kind of thing is interesting and makes tuning challenging. I see it less of an interpolation thing, and more of an interaction between two variables thing. Another example would be: a) increase rook on open file = +, b) increase rook mobility = +, a & b = - (because overall rook increase in value causes too many sacrifices of two pieces for rook and pawn). When it happens in interpolated variables figuring out the root issue certainly is more challenging though.

This kind of thing happens so much, it is one of the things I think causes so many programs to get stuck in local maxima, where improvement gets difficult.

-Sam

The ones that give me the most problems are the scores with a very "tight" usable range. I try a coarse "hunt" first, say 0, 50%, 100%, 150%, 200% where the percentage is a percent of the default score. Occasionally 0 is best meaning this term is worthless, and I have removed a few of those over the past month or so. But for some values, the optimal might be 75%, and by the time you drop to 50% or go past to 100%, the value looks bad. So if I run a test and see no change with those increments, either the value is worthless, or I skipped over the optimal range completely. That's harder to find, but has happened at least once...