Ab-initio evaluation tuning

[Evert]

The tuner now gives me the following set of evaluation parameters:

Code: Select all

VALUEL_P_MG          =   0.74 +/-   0.02 [190]
VALUEL_M_MG          =   3.03 +/-   0.09 [776]
VALUEL_R_MG          =   3.90 +/-   0.21 [998]
VALUEL_Q_MG          =   8.59 +/-   0.41 [2200]
VALUEL_P_EG          =   0.93 +/-   0.02 [239]
VALUEL_N_EG          =   3.04 +/-   0.06 [779]
VALUEL_B_EG          =   3.32 +/-   0.06 [850]
VALUEL_R_EG          =   6.08 +/-   0.11 [1556]
VALUEL_Q_EG          =  11.08 +/-   0.26 [2836]
VALUEQ_BB_MG         =   0.13 +/-   0.03 [33]
VALUEQ_BB_EG         =   0.23 +/-   0.04 [59]
VALUEQ_PASS_MG       =   0.71 +/-   0.18 [183]
VALUEQ_PASS_EG       =   3.11 +/-   0.09 [796]
VALUE_WIN_EG         =   4.44 +/-   0.15 [1136]
MOBILITY_N_MG        =   0.00 +/-   0.00 [1]
MOBILITY_N_EG        =   0.00 +/-   0.00 [0]
MOBILITY_B_MG        =   0.00 +/-   0.00 [0]
MOBILITY_B_EG        =  -0.00 +/-   0.00 [-0]
MOBILITY_R_MG        =  -0.00 +/-   0.00 [-1]
MOBILITY_R_EG        =   0.00 +/-   0.00 [1]
MOBILITY_Q_MG        =  -0.00 +/-   0.00 [-0]
MOBILITY_Q_EG        =  -0.00 +/-   0.01 [-1]
PST_CENTRE_P_MG      =   0.01 +/-   0.00 [3]
PST_CENTRE_N_MG      =   0.05 +/-   0.00 [12]
PST_CENTRE_B_MG      =   0.05 +/-   0.00 [12]
PST_CENTRE_Q_MG      =  -0.00 +/-   0.01 [-1]
PST_CENTRE_K_EG      =   0.04 +/-   0.00 [11]
PST_CENTRE_N_EG      =   0.08 +/-   0.01 [20]
PST_CENTRE_B_EG      =   0.05 +/-   0.01 [12]
PST_CENTRE_Q_EG      =   0.06 +/-   0.01 [16]

The Rook is still low, but not as low as it was. I combine Knight and Bishop MG values into a single "minor" value (the tuner sets them equal anyway when I don't).
The PST_* terms are just the weight for the centre in the PST (currently the only component in the tables) and are multiplied by the following table to get the piece-specific PST:

Code: Select all

  -7  -4  -2  -1  -1  -2  -4  -7
  -4  -1   1   2   2   1  -1  -4
  -2   1   3   4   4   3   1  -2
  -1   2   4   5   5   4   2  -1
  -1   2   4   5   5   4   2  -1
  -2   1   3   4   4   3   1  -2
  -4  -1   1   2   2   1  -1  -4
  -7  -4  -2  -1  -1  -2  -4  -7

What I find really surprising, however, is that the tuner always tunes the mobility weights down to 0. I thought this was due to the weights starting out at 0 and the tuner being stuck in the wrong minimum, but even if I start them out at some larger value, I end up back with the above. The mobility term is weight*(N - (<N>-1)), where N is the number of moves and <N> is the median number of moves for that piece type on an empty board. This is decreased by 1 to crudely take into account that the board is normally not empty (as such, it isn't a particularly good estimate). This gives the following progression for each piece type:

Code: Select all

Knight    :   2   5   4   8 {  -3  -2  -1   0   1   2   3   4   5}
Bishop    :   7   9   7  13 {  -6  -5  -4  -3  -2  -1   0   1   2   3   4   5   6   7}
Rook      :  14  14  14  14 {  -7  -6  -5  -4  -3  -2  -1   0   1   2   3   4   5   6   7}
Queen     :  21  23  21  27 { -20 -19 -18 -17 -16 -15 -14 -13 -12 -11 -10  -9  -8  -7  -6  -5  -4  -3  -2  -1   0   1   2   3   4   5   6   7}

(The four numbers are the minimum, medium, median and maximum number of moves on an empty board). As I said though, the actual mobility weights are set to 0 by the tuner.

Has anyone successfully managed to tune mobility weights using test positions? The only explanation I can think of is that it is the interaction with the search that makes mobility a valuable term, but I find that highly surprising. Thoughts?

[Evert]

Typical. I post about the issue, and I figure out what's up shortly afterwards. Turns out I forgot to subtract the White and Black contributions to the final score (I was adding them instead). With that fixed,

Code: Select all

MOBILITY_N_MG        =   0.02 +/-   0.01 [6]
MOBILITY_N_EG        =  -0.00 +/-   0.01 [-1]
MOBILITY_B_MG        =   0.07 +/-   0.01 [18]
MOBILITY_B_EG        =   0.07 +/-   0.01 [18]
MOBILITY_R_MG        =   0.06 +/-   0.00 [15]
MOBILITY_R_EG        =   0.02 +/-   0.00 [6]
MOBILITY_Q_MG        =   0.02 +/-   0.00 [4]
MOBILITY_Q_EG        =   0.07 +/-   0.01 [18]
PST_CENTRE_P_MG      =   0.01 +/-   0.00 [2]
PST_CENTRE_N_MG      =   0.03 +/-   0.01 [8]
PST_CENTRE_B_MG      =  -0.00 +/-   0.01 [-1]
PST_CENTRE_Q_MG      =  -0.03 +/-   0.01 [-7]
PST_CENTRE_K_EG      =   0.05 +/-   0.00 [12]
PST_CENTRE_N_EG      =   0.09 +/-   0.01 [23]
PST_CENTRE_B_EG      =   0.01 +/-   0.01 [1]
PST_CENTRE_Q_EG      =   0.04 +/-   0.01 [9]

So as not to make this completely useless, some further details on how I do the tuning run.
It should be fairly obvious, but it helps if the test positions used for tuning actually contain the features you're trying to tune, which is apparent in the condition number of the Jacobian. For piece values, this means unbalanced material. For mobility, this is of course not an issue (you can always tune mobility). As a bonus, things run faster.

When tuning, I split the positions in half (at random) and first tune using half the positions before doing it again on the larger set. My main motivation for doing this is speed (it's much faster to run on a smaller number of test positions, and if the solution is reasonable you converge more quickly on the final set as well), but it has the added benefit of acting as a sanity-check: you expect the parameters to not change very much between the two run.
I tried finer divisions, but they don't offer any speed advantage.