Side to Move Bonus---does it help?

[Jan Brouwer]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

I wonder if the side to move bonus at the end of a long PV is really comparable to the side to move bonus in a given position.
Anyhow, if your testing indicates that the bonus improves strength even when compensating for tree size differences,
then maybe it is a sign of a systematic bias in quiescence search (given that most quiescence searches are very short), caused by the standing pat part?

[Ryan Benitez]

Where is your mathematical proof that mobility works? Most programs use mobility because it seems to work well with their programs as decided by emperical tests. But mobility is just like every other evaluation feature, it's only a crude approximation of the goodness of some aspect of the position. Which means it is wrong sometimes.

I'm also not sure how mobility tells you that it's better to be on the move than not on the move. You are evaluating the same position, unless of course your evaluation takes into consideration who's turn to move it is, in which case you pretty much have implemented the side to move bonus via a different mechanism.

I think almost every strong program has a small side to move bonus including Rybka, a program much stronger than yours.

In simple terms mobility works because the more squares a piece can get to the more likely it is to be useful. The piece values we give pieces are very close in ratio to the long term mobility of the pieces. The long term mobility having a close ratio to the piece values should have been common sense but I went the rout of several gigs of log files to figure it out instead. The data from the log files is evidence, to me at least, on why mobility works. The thousands of games people have played with many engines using mobility is proof that it does work. The side to move bonus is an assumption that is somewhat right more often than not. It is a short term tempo fix waiting for a better solution.

[Don]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

I wonder if the side to move bonus at the end of a long PV is really comparable to the side to move bonus in a given position.
Anyhow, if your testing indicates that the bonus improves strength even when compensating for tree size differences,
then maybe it is a sign of a systematic bias in quiescence search (given that most quiescence searches are very short), caused by the standing pat part?

I don't understand why this isn't an obvious positional term in any chess program (in fact it is.)

It's well known that in the opening position white has a real advantage. Assuming there is no stand pat bonus in your program, a 1 ply search will usually return a big advantage for white. This is exactly how it should be unless your evaluation is broken.

But if you do the same search at 2 ply, the search will believe that the position is even. Having mobility in your program does not solve this problem as Vincent thinks, because both program can play their e pawn out and have the same exact mobility count.

Your program probably has all kinds of valid positional concepts expressed, such as advanced passed pawns, etc. All of these are just guesses, that we try to refine with more and more conditions to try to improve the quality of the evaluation function.

So how is it that the side to move is not a valid positional concept? Do you believe that given 2 identical positions, there is no advantage whatsoever to being the side whose turn it is to move? I don't think any reasonable chess player would argue that it isn't.

You have to keep in mind that most static evaluation functions return the same evaluation score regardless of whose turn to move it is, unless they have a stand pat bonus or some other mechanism.

Am I saying this is the only way to do it? No, like any other evaluation term it can be conditioned and improved on or represented in some other fashion. It's like rook to the 7th rank, a good rule in general, but in some situations it's totally irrelevant. A more sophisticated program will put conditions on this rule, but it's probably far better to have it the stupid way than not at all.

One thing my program has is a bonus for up-attacking the opponent as an evaluation feature. It's another imperfect rule but it recognizes the fact that if you are up-attacked, you probably are going to have to make a defensive move and it will be the opponents turn to move. So this bonus in my program can cancel the stand pat bonus. It's a tempo rule you might say, just as stand pat bonus is. The stand pat bonus rule cannot cover tactical tempo, but it covers positional tempo.

[wgarvin]

Suppose the actual end of the game is N moves in the future. The player with the move gets to move ceil(N/2) times, while the player without the move only gets to move floor(N/2) more times. If you average this out over a lot of game positions in a lot of situations where all other things are about equal (e.g. no zugzwang, material and positional strength about equal, etc), then on average, the player with the move would have slightly more opportunity to positively affect the outcome of the game by his choice of moves. Or at least, a slightly better chance of preventing his opponent from getting an advantage.

[bob]

Suppose the actual end of the game is N moves in the future. The player with the move gets to move ceil(N/2) times, while the player without the move only gets to move floor(N/2) more times. If you average this out over a lot of game positions in a lot of situations where all other things are about equal (e.g. no zugzwang, material and positional strength about equal, etc), then on average, the player with the move would have slightly more opportunity to positively affect the outcome of the game by his choice of moves. Or at least, a slightly better chance of preventing his opponent from getting an advantage.

I am convinced it works, because I tested it exhaustively. It is not a huge gain, but it is measurable. The values we use in Crafty were tuned on the cluster and made a measurable improvement. we are not talking +10 elo here, more like +4-5 max. But anything + is good if you get enough of them.

[Jan Brouwer]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

I wonder if the side to move bonus at the end of a long PV is really comparable to the side to move bonus in a given position.
Anyhow, if your testing indicates that the bonus improves strength even when compensating for tree size differences,
then maybe it is a sign of a systematic bias in quiescence search (given that most quiescence searches are very short), caused by the standing pat part?

I don't understand why this isn't an obvious positional term in any chess program (in fact it is.)

It's well known that in the opening position white has a real advantage. Assuming there is no stand pat bonus in your program, a 1 ply search will usually return a big advantage for white. This is exactly how it should be unless your evaluation is broken.

But if you do the same search at 2 ply, the search will believe that the position is even. Having mobility in your program does not solve this problem as Vincent thinks, because both program can play their e pawn out and have the same exact mobility count.

Your program probably has all kinds of valid positional concepts expressed, such as advanced passed pawns, etc. All of these are just guesses, that we try to refine with more and more conditions to try to improve the quality of the evaluation function.

So how is it that the side to move is not a valid positional concept? Do you believe that given 2 identical positions, there is no advantage whatsoever to being the side whose turn it is to move? I don't think any reasonable chess player would argue that it isn't.

You have to keep in mind that most static evaluation functions return the same evaluation score regardless of whose turn to move it is, unless they have a stand pat bonus or some other mechanism.

Am I saying this is the only way to do it? No, like any other evaluation term it can be conditioned and improved on or represented in some other fashion. It's like rook to the 7th rank, a good rule in general, but in some situations it's totally irrelevant. A more sophisticated program will put conditions on this rule, but it's probably far better to have it the stupid way than not at all.

One thing my program has is a bonus for up-attacking the opponent as an evaluation feature. It's another imperfect rule but it recognizes the fact that if you are up-attacked, you probably are going to have to make a defensive move and it will be the opponents turn to move. So this bonus in my program can cancel the stand pat bonus. It's a tempo rule you might say, just as stand pat bonus is. The stand pat bonus rule cannot cover tactical tempo, but it covers positional tempo.

It is obvious that the side to move bonus works from your and Bob's testing, it is just not obvious to me why (how) it works (other than by reducing tree sizes).

My program also shows the (diminishing) odd/even effect when iterating deeper, but again it is not obvious to me that avoiding this effect improves strength.

My thinking was that the stand pat rule of the quiescence search may, on average, be slightly pessimistic and can be compensated for by the side to move bonus. Your argument, if I understand it correctly, is that the evaluation function is imperfect in that the side to have made the move last has on average an advantage which can be compensated by the side to move bonus. And your tactical tempo is somewhere in between.

Evaluation and quiescence search are closely related anyhow, and there I think improvements (in the way they interact) can still be made (in my program certainly ).

[bob]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

I wonder if the side to move bonus at the end of a long PV is really comparable to the side to move bonus in a given position.
Anyhow, if your testing indicates that the bonus improves strength even when compensating for tree size differences,
then maybe it is a sign of a systematic bias in quiescence search (given that most quiescence searches are very short), caused by the standing pat part?

I don't understand why this isn't an obvious positional term in any chess program (in fact it is.)

It's well known that in the opening position white has a real advantage. Assuming there is no stand pat bonus in your program, a 1 ply search will usually return a big advantage for white. This is exactly how it should be unless your evaluation is broken.

But if you do the same search at 2 ply, the search will believe that the position is even. Having mobility in your program does not solve this problem as Vincent thinks, because both program can play their e pawn out and have the same exact mobility count.

Your program probably has all kinds of valid positional concepts expressed, such as advanced passed pawns, etc. All of these are just guesses, that we try to refine with more and more conditions to try to improve the quality of the evaluation function.

So how is it that the side to move is not a valid positional concept? Do you believe that given 2 identical positions, there is no advantage whatsoever to being the side whose turn it is to move? I don't think any reasonable chess player would argue that it isn't.

You have to keep in mind that most static evaluation functions return the same evaluation score regardless of whose turn to move it is, unless they have a stand pat bonus or some other mechanism.

Am I saying this is the only way to do it? No, like any other evaluation term it can be conditioned and improved on or represented in some other fashion. It's like rook to the 7th rank, a good rule in general, but in some situations it's totally irrelevant. A more sophisticated program will put conditions on this rule, but it's probably far better to have it the stupid way than not at all.

One thing my program has is a bonus for up-attacking the opponent as an evaluation feature. It's another imperfect rule but it recognizes the fact that if you are up-attacked, you probably are going to have to make a defensive move and it will be the opponents turn to move. So this bonus in my program can cancel the stand pat bonus. It's a tempo rule you might say, just as stand pat bonus is. The stand pat bonus rule cannot cover tactical tempo, but it covers positional tempo.

It is obvious that the side to move bonus works from your and Bob's testing, it is just not obvious to me why (how) it works (other than by reducing tree sizes).

My program also shows the (diminishing) odd/even effect when iterating deeper, but again it is not obvious to me that avoiding this effect improves strength.

My thinking was that the stand pat rule of the quiescence search may, on average, be slightly pessimistic and can be compensated for by the side to move bonus. Your argument, if I understand it correctly, is that the evaluation function is imperfect in that the side to have made the move last has on average an advantage which can be compensated by the side to move bonus. And your tactical tempo is somewhere in between.

Evaluation and quiescence search are closely related anyhow, and there I think improvements (in the way they interact) can still be made (in my program certainly ).

Let me reiterte my opinion on this subject. Search tricks reduce the size of the tree. Additions to the eval typically _increase_ the size. Here are a couple of positions. In each pair of node counts, the first line is with the normal wtm bonus set, the second is with both mg/eg wtm bonuses set to zero.

log.001: time=5.12 mat=0 n=9388329 fh=93% nps=1.8M
log.002: time=1.86 mat=0 n=3662306 fh=94% nps=2.0M

log.001: time=3.20 mat=0 n=6406922 fh=92% nps=2.0M
log.002: time=5.65 mat=0 n=11432660 fh=92% nps=2.0M

log.001: time=19.06 mat=0 n=28143964 fh=91% nps=1.5M
log.002: time=10.07 mat=0 n=15507694 fh=91% nps=1.5M

log.001: time=1.72 mat=0 n=4891428 fh=95% nps=2.8M
log.002: time=1.51 mat=0 n=4503212 fh=96% nps=3.0M

log.001: time=7.43 mat=0 n=16998198 fh=93% nps=2.3M
log.002: time=4.53 mat=0 n=10482423 fh=93% nps=2.3M


If you look, 4 out of 5 tests produced _larger_ trees with the wtm bonus included, one was smaller with the non-zero wtm bonus.

I've found that in general, any addition to the eval makes the tree _larger_. because it spreads the eval values out more which produces fewer cutoffs. In any case, for my program, it plays better, but the trees are larger for some cases. The above are just the last 5 positions from the kopec test. I can test most any position you want.

[diep]

Here's what's wrong with the idea. If you infer that smaller trees are better, get rid of all extensions. And any eval term that produces really large positional score bonuses. The tree size will go down. In fact, if you get rid of your evaluation completely, you will search really compact trees. And for tactical positions, this will do better in terms of faster solutions and smaller trees, but the program won't exactly play very well...

I have seen literally hundreds of cases where an eval change makes the tree larger, about as many as I have seen where an eval change makes the tree smaller. I don't think this is a good way to choose eval parameters. At least IMHO.

Tree search space is a function of several variables. If your eval change can somehow eliminate some searching, you might get good results for tuning a value. But many terms do not make the tree smaller, but do make the program stronger.

I agree for the general case, but for this bonus I think this method is valid.
It is such a simple feature, using only a single binary input, it is hard to see how it could have any influence on playing strength other than via tree size.
Perhaps a whole class of evaluation features can be found which can be optimized via tree sizes.
You could test this on your cluster, Bob, and provide those of us with more modest resources with a list of features we can actually tune

Hmm, I could actually test that myself I guess for those parameters in Crafty which have already been tuned...

In my testing the side to move bonus has a positive affect on playing strength, even when you ignore tree size considerations. It's very small, but it is an improvement probably because it's a valid concept - given some random chess position it is better for it to be your turn to move 99% of the time.

where is your mathematical proof of it?

you can sure write down some stuff as this, in contradiction to most eval terms is cooperating with search in a direct manner.

you can from there deduce that factual a side to move bonus is a form of mobility. now mobility usually works brilliant, yet if you did do quite some effort for your mobility how can a side to move bonus work?

Vincent

Where is your mathematical proof that mobility works? Most programs use mobility because it seems to work well with their programs as decided by emperical tests. But mobility is just like every other evaluation feature, it's only a crude approximation of the goodness of some aspect of the position. Which means it is wrong sometimes.

I'm also not sure how mobility tells you that it's better to be on the move than not on the move. You are evaluating the same position, unless of course your evaluation takes into consideration who's turn to move it is, in which case you pretty much have implemented the side to move bonus via a different mechanism.

I think almost every strong program has a small side to move bonus including Rybka, a program much stronger than yours.

The concept of proof that mobility works is very simple. When i'm winning bigtime i capture all your pieces, just our king gets left and when it is in the corner and mated its mobility is at a minimum and mine is maximal.

Now it's your turn for side to move bonus

Vincent

Let us play a game, but it's always my turn to move. Since you do not believe that having the move is an advantage, then you should easily win since you are a stronger player than I am. Nevertheless, I shall beat you in 4 moves.

And in the next tournament you play in, refuse the white pieces, always take the black side. Empirical proof says you would be at a disadvantage.

Let's stop this nonsense now.

If white makes 4 moves from openings position and the only form of evaluation you have is the side to move bonus at the leaf:

e4 Qh5 Bc4 Qxf7 mate, then you evaluate that leaf as side to move bonus +0.05 for black, as black has the move.

I'm evaluating it as ugly bad for black as he has no more mobility as it is a mate.

So side to move bonus messes up here, as it can't distinguish a near mate position between the position after 1.d4 which also is +0.05 for black.

Note that the claimed bonuses here are just very tiny. 0.05 of a pawn i saw circulate here up to 0.09, whereas mobility is doing the same thing yet a lot better and gives a much larger bonus/penalty and more accurate.

Vincent

[Kempelen]

I add side to move bonus, but with a limitation: Don't if in-check: Sometimes, quiesce end in a check position, so it is a kind of non-calm position.
Another idea to test is if in-check give the bonus to opponent.

[diep]

I add side to move bonus, but with a limitation: Don't if in-check: Sometimes, quiesce end in a check position, so it is a kind of non-calm position.
Another idea to test is if in-check give the bonus to opponent.

side to move BONUS is too inaccurate as a heuristic to be used IMHO and it is contraproductive for quiscencesearch i feel.

During extensive tests a year or 10 ago performed it lost quite some elo. At that time i did give a 0.10 bonus in non-endgame positions and no bonus in endgame. In endgame giving bonus was losing that much elo that i very soon had to stop it. Note that being in check can never be a leaf in Diep so that's not a problem either.

There is a huge difference IMHO between chessknowledge that nearly always works, like a white bishop on d3 blocking a pawn on d2, versus heuristic that doesn't work in crucial positions where our form of qsearch already fails miserably. We increase in dramatic manner the mistake that qsearch already gives to our search.

Moreover, if we would replace the side to move bonus by mobility for the side to move, it already dramatically would improve the heuristic. Yet that still doesn't solve the fact that asymmetric bonus there is increasing the quiescencesearch error.

Claims that side to move bonus works in actual slower time controls would be amazing to me, as some years ago during tests i wasn't the only one concluding it didn't work for Diep.

As for the claim that it would compensate for the reduction effect, that is of course utter ballony, as you're also giving that bonus in non-reduced lines and transposition cutoffs would mess it up anyway. Additionally to that the bonus in reduced lines would increase the horizon effect that reductions cause.

So i assume that most claims here is by those who test in inaccurate manner or at too fast time controls (anything works then).

Vincent