Side to Move Bonus---does it help?

[Osipov Jury]

First, if this is from Rybka, apparently if I read this code right, for Rybka the SideToMoveBonus goes down towards the endgame, not up. In Crafty it is the opposite?

Yes.

Second, it is strange to see such a coarse resolution of phase in Rybka. Maybe this is only done for the SideToMoveBonus?

No, not only for SideToMoveBonus.

[mcostalba]

I don't think Glaurung or Stockfish scale their SideToMoveBonus yet.

No, not yet, but if Rybka does this then probably we should either

[Jan Brouwer]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

[Jan Brouwer]

seems to me that this part:
(int)((1.0 - phase) * 0.0
is always going to equal zero. So why bother with that part of the calculation?

I think that this for one purpose: you can change STM_Endgame from 0.0 to another value.

Yes, it is the formula for the weighted average of two values, of which one happens to be 0.
Phase should then be [0.0, 1.0], so it assumes a maximum of 2Q + 2R + 2N + 2B for both sides I guess.

[bob]

Perhaps the best realization of this issue (from one very strong engine):

stage = majors * 2 + minors;
phase = ((double)stage) / 24.0;
SideToMoveBonus = (int)((1.0 - phase) * 0.0 + phase * 9.0);

I leave it to the moderators if this can be discussed.

My attempt at a contribution:

♦ First, if this is from Rybka, apparently if I read this code right, for Rybka the SideToMoveBonus goes down towards the endgame, not up. In Crafty it is the opposite?

In crafty it is certainly the opposite of the above. We tuned these two values over a million games or so, although small changes in the two values certainly make _very_ small changes in the Elo. You can easily go too large and wreck things..

♦ Second, it is strange to see such a coarse resolution of phase in Rybka. Maybe this is only done for the SideToMoveBonus? I don't see why Vas would have to do that, unless it is done to smoothen the bonus over the phase of the game and this is somehow important. But at an exchange of pieces you have a major discontinuity anyhow, there is nothing you can do about it, I don't see a jump in the side to move bonus is going to make that any worse and why you would not use a more detailed calculation of phase.

♦ Mr. Rajlich, once more you seem to be underestimating the potential material value of the Bishop Pair. Of course this is never going to work Now I know that for Rybka that involves at least three additions, the dark-squared bishop, the light squared bishop and the bishop pair material values but this is critical stuff Mr. Rajlich

In Ancalagon I don't have different material values for the dark squared bishop and the light squared bishop, but I have tried to include the value of the Bishop Pair in Tord's calculation of the game phase. It is not perfect because there can be potential conflicts in other places where it is assumed the phase only calculates the midgame material values. But so far I did not spot a major conflict in the rest of Glaurung's code or that of Stockfish when I do this. I don't think Glaurung or Stockfish scale their SideToMoveBonus yet.

This is new Ancalagon code with 'minor' changes from Glaurung:

Code: Select all

/// Position::compute_non_pawn_material() computes the total non-pawn middle
/// game material score for the given side. Material scores are updated
/// incrementally during the search, this function is only used while
/// initializing a new Position object.

Value Position::compute_non_pawn_material(Color c) const {

  Value result = Value(0);

  for (PieceType pt = KNIGHT; pt <= QUEEN; pt++)
  {
      Bitboard b = pieces_of_color_and_type(c, pt);
      int Bishopcount = 0;
      while (b)
      {
          assert(piece_on(first_1(b)) == piece_of_color_and_type(c, pt));
          pop_1st_bit(&b);
          result += piece_value_midgame(pt);
          if (pt == BISHOP) Bishopcount++;
      }
      if (Bishopcount >= 2) result += Value(100);
  }
  return result;
}

Eelco

[Don]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

[diep]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

where is your mathematical proof of it?

you can sure write down some stuff as this, in contradiction to most eval terms is cooperating with search in a direct manner.

you can from there deduce that factual a side to move bonus is a form of mobility. now mobility usually works brilliant, yet if you did do quite some effort for your mobility how can a side to move bonus work?

Vincent

[Don]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

where is your mathematical proof of it?

you can sure write down some stuff as this, in contradiction to most eval terms is cooperating with search in a direct manner.

you can from there deduce that factual a side to move bonus is a form of mobility. now mobility usually works brilliant, yet if you did do quite some effort for your mobility how can a side to move bonus work?

Vincent

Where is your mathematical proof that mobility works? Most programs use mobility because it seems to work well with their programs as decided by emperical tests. But mobility is just like every other evaluation feature, it's only a crude approximation of the goodness of some aspect of the position. Which means it is wrong sometimes.

I'm also not sure how mobility tells you that it's better to be on the move than not on the move. You are evaluating the same position, unless of course your evaluation takes into consideration who's turn to move it is, in which case you pretty much have implemented the side to move bonus via a different mechanism.

I think almost every strong program has a small side to move bonus including Rybka, a program much stronger than yours.

[diep]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

where is your mathematical proof of it?

you can sure write down some stuff as this, in contradiction to most eval terms is cooperating with search in a direct manner.

you can from there deduce that factual a side to move bonus is a form of mobility. now mobility usually works brilliant, yet if you did do quite some effort for your mobility how can a side to move bonus work?

Vincent

Where is your mathematical proof that mobility works? Most programs use mobility because it seems to work well with their programs as decided by emperical tests. But mobility is just like every other evaluation feature, it's only a crude approximation of the goodness of some aspect of the position. Which means it is wrong sometimes.

I'm also not sure how mobility tells you that it's better to be on the move than not on the move. You are evaluating the same position, unless of course your evaluation takes into consideration who's turn to move it is, in which case you pretty much have implemented the side to move bonus via a different mechanism.

I think almost every strong program has a small side to move bonus including Rybka, a program much stronger than yours.

The concept of proof that mobility works is very simple. When i'm winning bigtime i capture all your pieces, just our king gets left and when it is in the corner and mated its mobility is at a minimum and mine is maximal.

Now it's your turn for side to move bonus

Vincent

[Don]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

where is your mathematical proof of it?

you can sure write down some stuff as this, in contradiction to most eval terms is cooperating with search in a direct manner.

you can from there deduce that factual a side to move bonus is a form of mobility. now mobility usually works brilliant, yet if you did do quite some effort for your mobility how can a side to move bonus work?

Vincent

Where is your mathematical proof that mobility works? Most programs use mobility because it seems to work well with their programs as decided by emperical tests. But mobility is just like every other evaluation feature, it's only a crude approximation of the goodness of some aspect of the position. Which means it is wrong sometimes.

I'm also not sure how mobility tells you that it's better to be on the move than not on the move. You are evaluating the same position, unless of course your evaluation takes into consideration who's turn to move it is, in which case you pretty much have implemented the side to move bonus via a different mechanism.

I think almost every strong program has a small side to move bonus including Rybka, a program much stronger than yours.

The concept of proof that mobility works is very simple. When i'm winning bigtime i capture all your pieces, just our king gets left and when it is in the corner and mated its mobility is at a minimum and mine is maximal.

Now it's your turn for side to move bonus

Vincent

Let us play a game, but it's always my turn to move. Since you do not believe that having the move is an advantage, then you should easily win since you are a stronger player than I am. Nevertheless, I shall beat you in 4 moves.

And in the next tournament you play in, refuse the white pieces, always take the black side. Empirical proof says you would be at a disadvantage.

Let's stop this nonsense now.