Smooth scaling -- an explanation

[Dann Corbit]

Consider the following:
#ifdef SMOOTH_REDUCTION
double delta = approximateEval - beta;
delta = max(delta, 1.0);
double ddepth = double(depth);
double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;
r = r > ddepth ? ddepth : r;
int R = int(r);
#else
// Null move dynamic reduction based on depth
int R = (depth >= 5 * OnePly ? 4 : 3);

// Null move dynamic reduction based on value
if (approximateEval - beta > PawnValueMidgame)
R++;
#endif

If depth = 5, then 0.18 * 5 = 0.9
So 0.9 + 3.1 gives 4.0 which is the same as the original formula.
In value.h, we find this:
const Value PawnValueMidgame = Value(0x0C6);
In decimal, C6 is 198.
log (base e) of 198 is 5.288
dividing 5.288 by 5 gives 1 (approximately).

So, for the points at the boundary conditions of the old formula, my curve gives the same answers (or answers pretty nearly). When you fit this idea to your code, I suggest that you calibrate the curve accordingly, especially if you have carefully weighed out your null move reduction constants by experiment.

The difference is that when we bend away from these answers the solution I propose slowly changes so that instead of slamming the door, we are slowly closing it.

The reason I came up with smooth scaling is that null move's abrupt nature really bothered me. Why do we reduce the same in a 30 ply search as we do in a 12 ply search? That does not make sense. Why do we reduce the same with a full queen advantage as with a knight advantage? That also seemed very strange. So I just did a simple extrapolation to smooth out the reductions in order that they made sense to me.

[michiguel]

Consider the following:
#ifdef SMOOTH_REDUCTION
double delta = approximateEval - beta;
delta = max(delta, 1.0);
double ddepth = double(depth);
double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;
r = r > ddepth ? ddepth : r;
int R = int(r);
#else
// Null move dynamic reduction based on depth
int R = (depth >= 5 * OnePly ? 4 : 3);

// Null move dynamic reduction based on value
if (approximateEval - beta > PawnValueMidgame)
R++;
#endif

If depth = 5, then 0.18 * 5 = 0.9
So 0.9 + 3.1 gives 4.0 which is the same as the original formula.
In value.h, we find this:
const Value PawnValueMidgame = Value(0x0C6);
In decimal, C6 is 198.
log (base e) of 198 is 5.288
dividing 5.288 by 5 gives 1 (approximately).

So, for the points at the boundary conditions of the old formula, my curve gives the same answers (or answers pretty nearly). When you fit this idea to your code, I suggest that you calibrate the curve accordingly, especially if you have carefully weighed out your null move reduction constants by experiment.

The difference is that when we bend away from these answers the solution I propose slowly changes so that instead of slamming the door, we are slowly closing it.

The reason I came up with smooth scaling is that null move's abrupt nature really bothered me. Why do we reduce the same in a 30 ply search as we do in a 12 ply search? That does not make sense. Why do we reduce the same with a full queen advantage as with a knight advantage? That also seemed very strange. So I just did a simple extrapolation to smooth out the reductions in order that they made sense to me.

This is for the non-believers and deniers

First of all, it needs to be tested, but some indicated that it should not work (based on intuition and/or theoretical issues). I disagree. It is wrong to say that if null move alone gives you 80 points, tweaking null move cannot give you more than, say, 20 points. It is possible to tweak nullmove and obtain a bigger boost than the addition of the technique itself... at least in theory...

Lets say that nullmove allows you to search 3 plies deeper. 3 plies deeper may mean 300 ELO points. Why do you see an increase of only 100 points? Because the search is full of holes. So, in this case, there is 200 points to be gained by avoiding the holes. It is as important to know WHEN to do cut off by nullmove as to the reduction you get. It is possible that Dann's adjustment may cause a more careful decision making at the level of nullmove for certain engines. I could see that this could be completely dependent on the engine.

and BTW, this is not adaptive null move, it has nothing to do with the snippet from Robbert Litto, and who the heck cares whether someone whisper a similar thought about this 5 years ago. Drop y'all EGO points and test to see whether it gives ELO points.

In experimental science you learn that ideas are meaningless. My former PhD's advisor said to me, everybody has plenty, an most likely, yours is not original. Someone else is thinking about it or already did . What matters is the execution and who finally did it. What my PhD advisor was trying to say was (he was too polite to say it directly), "get your ass to the bench and start doing the experiment"

Miguel

[mcostalba]

If depth = 5, then 0.18 * 5 = 0.9
So 0.9 + 3.1 gives 4.0 which is the same as the original formula.

No, in the original is (depth >= 5 * OnePly ? 4 : 3) and OnePly == 2, so you erroneusly multiplied all by 2 and you got a surprise !

Just to be clear depth = 5 it means depth = 2,5*OnePly and reduction is 3 not 4 in the original formula.

I think you missed a factor of 2 along the way, when I have done with 1.6.2 I will setup a table with a proper comparison of different reductions per depth between 1.6 and your version....

[guillef]

Following Marco's comments,

the expression can be corrected by:

double delta = eval - beta;
delta = max(delta, 1.0);
double ddepth = double(depth/OnePly);
double r = 0.18 * ddepth + 3.1 + log(delta)/5;
r = r > ddepth ? ddepth : r;
int R = int(r * OnePly);

.. and OnPly can be 2, 4 or any other value.

BTW, if PawnValue is 100 and not 198 (like in stockfish) "log(delta)/5" can be modified by "log(delta)/4.5".

[mcostalba]

Yes,

the expression can be corrected by:

But currently is not corrected, and I think that the reason it reaches two plies more is exactly because of this.

I will analyze why this "blunder" produced an interesting result....but it _is_ a blunder because the author meant a much smaller reduction then what in reality is, as he explained clearly above.

[Dann Corbit]

If depth = 5, then 0.18 * 5 = 0.9
So 0.9 + 3.1 gives 4.0 which is the same as the original formula.

No, in the original is (depth >= 5 * OnePly ? 4 : 3) and OnePly == 2, so you erroneusly multiplied all by 2 and you got a surprise !

Just to be clear depth = 5 it means depth = 2,5*OnePly and reduction is 3 not 4 in the original formula.

I think you missed a factor of 2 along the way, when I have done with 1.6.2 I will setup a table with a proper comparison of different reductions per depth between 1.6 and your version....

I printed my reductions alongside the originals and they came out about the same. I am not sure if I follow what you think I did wrong though.

[guillef]

Yes,

the expression can be corrected by:

But currently is not corrected, and I think that the reason it reaches two plies more is exactly because of this.

I will analyze why this "blunder" produced an interesting result....but it _is_ a blunder because the author meant a much smaller reduction then what in reality is, as he explained clearly above.

I run a tourney with ORIGINAL formula in chronos an it scores 60% over best previous version!!

In case that original formula is the best option (even more than corrected formula) the next line must be corrected:

r = r > ddepth ? ddepth : r;

because here r are "pure" depth and ddepth are "fractional adapted" depth.

[Dann Corbit]

If depth = 5, then 0.18 * 5 = 0.9
So 0.9 + 3.1 gives 4.0 which is the same as the original formula.

No, in the original is (depth >= 5 * OnePly ? 4 : 3) and OnePly == 2, so you erroneusly multiplied all by 2 and you got a surprise !

Just to be clear depth = 5 it means depth = 2,5*OnePly and reduction is 3 not 4 in the original formula.

I think you missed a factor of 2 along the way, when I have done with 1.6.2 I will setup a table with a proper comparison of different reductions per depth between 1.6 and your version....

I printed my reductions alongside the originals and they came out about the same. I am not sure if I follow what you think I did wrong though.

Daniel Shawul also printed a table of the reductions that you can examine somewhere in this forum

[mcostalba]

I printed my reductions alongside the originals and they came out about the same. I am not sure if I follow what you think I did wrong though.

I have to dedicate some serious time to analyze what your alghortim does and what does 1.6 original. Until now I didn't had that time, hope to find in the next days.

Please, until then, consider just a bit of handwaving what I write.

Regarding your explanation I was just saying that when you write in your example depth = 5, actually it is depth = 2,5*OnePly so that when you say that with depth = 5 you have the same reduction by 4 of the original, this is not correct because the original with a depth = 5 (aka 2,5*OnePly) gets a reduction of 3 not 4.

[Dann Corbit]

I printed my reductions alongside the originals and they came out about the same. I am not sure if I follow what you think I did wrong though.

I have to dedicate some serious time to analyze what your alghortim does and what does 1.6 original. Until now I didn't had that time, hope to find in the next days.

Please, until then, consider just a bit of handwaving what I write.

Regarding your explanation I was just saying that when you write in your example depth = 5, actually it is depth = 2,5*OnePly so that when you say that with depth = 5 you have the same reduction by 4 of the original, this is not correct because the original with a depth = 5 (aka 2,5*OnePly) gets a reduction of 3 not 4.

Yes, I knew the meaning of OnePly, but I just calibrated to the same equation as the original (or so I thought). If I really was off by a factor of 2, I guess that the modified program couldn't punch its way out of a wet paper bag, but sometimes great things do come from accidents.

At any rate, I guess that the fundamental idea will lead to careful experiments that result in something good in the long run.

It was never intended as a final update for some formal version of Stockfish. To paraphrase, I meant it as this:
"Hey, look at this idea ... Now how can we improve it?"