Smooth scaling stockfish

[Gian-Carlo Pascutto]

I doubt that 150 Elo will hold, but I guess at least 100 Elo will. Then again, my test was not very extensive.

For me it seems 15 ELO worse than regular nullmove.

[BubbaTough]

This version of Stockfish:
http://cap.connx.com/chess-engines/new- ... x64_ja.rar

Scales null move smoothly. I get about +150 Elo so far in my testing. How about yours?

I never got that idea to work, but I did test it quite a few years ago when depths were not as significant. In fact, Ernst and I developed the "adaptive null-move idea" independently. John Stanback had suggested the basics for the idea around 1995 or so. Initially it was a question of R=1 or R=2, but eventually became R=2 or R=3. I used to have fractional plies, and tried a gradual and continuous (smooth) reduction as it approached the frontier nodes, but never found something that worked any better than the simple R=3 closer to the root, R=2 closer to the leaves.

With Stockfish 1.6, I get a branching factor of well under 2. It clearly adds strength. I guess that when the idea is polished, it will be a lot better. I have other experiments that do something similar (well, actually the opposite as far as pruning goes -- so I guessed it would help) but I thought a quick demonstration of the idea might benefit the chess community.

I haven't looked but what is the "range"? I used to use 3 down to 2, but when I added the qsearch check and check-evasion code, I went with 3 everywhere. If they reduce more than 3, I could see a possible benefit...

It sounds like to me you really haven't tried this idea before. It is not at all like the "adaptive null-move" idea from 1995 in my mind. Perhaps you should look at the code, it is not complicated. It looks like a variation of the adaptive depth Sherwin talked about a couple years ago (I can't really remember exactly what he did, maybe something similar to nullDepth = 2+depth/3 or something). The version Corbit proposes here is more similar to what I eventually settled on. Its a little sad to see the concept get so much more press when added to a strong engine, than when the depth dependent idea was introduced into a weaker engine like Romi, but perhaps the stronger emphasis in Corbit's version on delta contributes enough to distinguish itself from Sherwin's concepts.

-Sam

[bob]

This version of Stockfish:
http://cap.connx.com/chess-engines/new- ... x64_ja.rar

Scales null move smoothly. I get about +150 Elo so far in my testing. How about yours?

I never got that idea to work, but I did test it quite a few years ago when depths were not as significant. In fact, Ernst and I developed the "adaptive null-move idea" independently. John Stanback had suggested the basics for the idea around 1995 or so. Initially it was a question of R=1 or R=2, but eventually became R=2 or R=3. I used to have fractional plies, and tried a gradual and continuous (smooth) reduction as it approached the frontier nodes, but never found something that worked any better than the simple R=3 closer to the root, R=2 closer to the leaves.

With Stockfish 1.6, I get a branching factor of well under 2. It clearly adds strength. I guess that when the idea is polished, it will be a lot better. I have other experiments that do something similar (well, actually the opposite as far as pruning goes -- so I guessed it would help) but I thought a quick demonstration of the idea might benefit the chess community.

I haven't looked but what is the "range"? I used to use 3 down to 2, but when I added the qsearch check and check-evasion code, I went with 3 everywhere. If they reduce more than 3, I could see a possible benefit...

The formula is

double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;

delta is at least 1 so the last component is not negative and it means if I understand correctly that they reduce at least 2.1 plies+18% of the remaining depth(the number in the formula is 3.1 and not 2.1 but there is 1 ply that you always reduce so I do not count it)
Uri

Is this from the 1.6 source? I don't see anything like that in the source I just downloaded. All I see with respect to the reduction amount is this:

Code: Select all

        // Null move dynamic reduction based on depth
        int R = (depth >= 5 * OnePly ? 4 : 3);

        // Null move dynamic reduction based on value
        if (approximateEval - beta > PawnValueMidgame)
            R++;

or was this code that Dann added himself???

[bob]

+150elo???

If we can get even close to this, I think that this is really great news for whole chess community!!

Code: Select all

		double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;

Balancing this equation is perhaps the next challenge? I need to think about this...

I think that it may be interesting to balance this equation also with
replacing approximateEval = quick_evaluate(pos) by something that is closer to the real evaluation or the real evaluation.

Even if evaluate(pos) is too expensive to calculate then it is not expensive to calculate average difference between quick_evaluate() and evaluate()
for the cases that evaluate() is being calculated and later use the average value for better approximation.

Uri

I find this _really_ hard to swallow myself. If I turn null-move completely off, it is an 80 Elo reduction over 40,000 games. I find it difficult to imagine how any scaling is going to make the idea 2x _more_ efficient.

Unfortunately, I am still waiting on the A/C repairs to happen and can't test this, but once the cluster comes up I can discover exactly what effect this has on Stockfish since I already use it in my cluster testing.

[bob]

+150elo???

If we can get even close to this, I think that this is really great news for whole chess community!!

Code: Select all

		double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;

Balancing this equation is perhaps the next challenge? I need to think about this...

I think that it may be interesting to balance this equation also with
replacing approximateEval = quick_evaluate(pos) by something that is closer to the real evaluation or the real evaluation.

Even if evaluate(pos) is too expensive to calculate then it is not expensive to calculate average difference between quick_evaluate() and evaluate()
for the cases that evaluate() is being calculated and later use the average value for better approximation.

Uri

Yes, there are many possible tweaks, also calculate the evaluation and use it could be an option to test.

But as usual is far more quick to get the idea then to test it

I think here what is needed most is testing...ideas are not a problem


If the first cut of this as has been posted by Dann is shown to work then, if we really want to converge quickly to an optimum solution for the benefit of the comunity we should use Bob's cluster..... if he agrees

When I can run again, I'll certainly be willing to do some testing, if this really pans out to be a gain. Still no ETA on the A/C parts so that we can power the cluster back up, but it ought to be pretty soon now, as this downtime is bordering on ridiculous...

[Karlo Bala]

This version of Stockfish:
http://cap.connx.com/chess-engines/new- ... x64_ja.rar

Scales null move smoothly. I get about +150 Elo so far in my testing. How about yours?

Something similar was done in rob*****:

Code: Select all

        null_do();
        hole_new = DEPTH - null_riduzione; 
        hole_new -= ((uint32)(MINIMUM(POSITION_0->value - VALUE, 96))) / 32;

[BubbaTough]

Is this from the 1.6 source? I don't see anything like that in the source I just downloaded. All I see with respect to the reduction amount is this:

Code: Select all

        // Null move dynamic reduction based on depth
        int R = (depth >= 5 * OnePly ? 4 : 3);

        // Null move dynamic reduction based on value
        if (approximateEval - beta > PawnValueMidgame)
            R++;

or was this code that Dann added himself???

Ahh good question. I ran into the same issue. This is Dan's change, not reflected in that source code (he posted a link to it elsewhere after I requested it):

Code: Select all

#ifdef SMOOTH_REDUCTION
		double delta = approximateEval - beta;
		delta = max(delta, 1.0);
		double ddepth = double(depth);
		double r = 0.18 * ddepth + 3.1 + log(delta)/5.0;
		r = r > ddepth ? ddepth : r;
		int R = int(r);
#else
        // Null move dynamic reduction based on depth
        int R = (depth >= 5 * OnePly ? 4 : 3);

        // Null move dynamic reduction based on value
        if (approximateEval - beta > PawnValueMidgame)
            R++;
#endif

-Sam

[Daniel Shawul]

Here is a table of values of the result of the formula.
First column is depth, second column margin (differnce between beta and eval), last column is R (including the normal 1 unit reduction).

Code: Select all

3	50	3.00	5	50	4.34	7	50	4.70	9	50	5.06	11	50	5.42	13	50	5.78
3	100	3.00	5	100	4.40	7	100	4.76	9	100	5.12	11	100	5.48	13	100	5.84
3	150	3.00	5	150	4.44	7	150	4.80	9	150	5.16	11	150	5.52	13	150	5.88
3	200	3.00	5	200	4.46	7	200	4.82	9	200	5.18	11	200	5.54	13	200	5.90
3	250	3.00	5	250	4.48	7	250	4.84	9	250	5.20	11	250	5.56	13	250	5.92
3	300	3.00	5	300	4.50	7	300	4.86	9	300	5.22	11	300	5.58	13	300	5.94
3	350	3.00	5	350	4.51	7	350	4.87	9	350	5.23	11	350	5.59	13	350	5.95
3	400	3.00	5	400	4.52	7	400	4.88	9	400	5.24	11	400	5.60	13	400	5.96
3	450	3.00	5	450	4.53	7	450	4.89	9	450	5.25	11	450	5.61	13	450	5.97
3	500	3.00	5	500	4.54	7	500	4.90	9	500	5.26	11	500	5.62	13	500	5.98
3	550	3.00	5	550	4.55	7	550	4.91	9	550	5.27	11	550	5.63	13	550	5.99
4	50	4.00	6	50	4.52	8	50	4.88	10	50	5.24	12	50	5.60	14	50	5.96
4	100	4.00	6	100	4.58	8	100	4.94	10	100	5.30	12	100	5.66	14	100	6.02
4	150	4.00	6	150	4.62	8	150	4.98	10	150	5.34	12	150	5.70	14	150	6.06
4	200	4.00	6	200	4.64	8	200	5.00	10	200	5.36	12	200	5.72	14	200	6.08
4	250	4.00	6	250	4.66	8	250	5.02	10	250	5.38	12	250	5.74	14	250	6.10
4	300	4.00	6	300	4.68	8	300	5.04	10	300	5.40	12	300	5.76	14	300	6.12
4	350	4.00	6	350	4.69	8	350	5.05	10	350	5.41	12	350	5.77	14	350	6.13
4	400	4.00	6	400	4.70	8	400	5.06	10	400	5.42	12	400	5.78	14	400	6.14
4	450	4.00	6	450	4.71	8	450	5.07	10	450	5.43	12	450	5.79	14	450	6.15
4	500	4.00	6	500	4.72	8	500	5.08	10	500	5.44	12	500	5.80	14	500	6.16
4	550	4.00	6	550	4.73	8	550	5.09	10	550	5.45	12	550	5.81	14	550	6.17

[mjlef]

This version of Stockfish:
http://cap.connx.com/chess-engines/new- ... x64_ja.rar

Scales null move smoothly. I get about +150 Elo so far in my testing. How about yours?

How about some details of your testing? Like number of games, opponents, time controls, of versions both before and after the change?

[mcostalba]

Here is a table of values of the result of the formula.
First column is depth, second column margin (differnce between beta and eval), last column is R (including the normal 1 unit reduction).

It seems very similar to the current formula (PawnValueMidgame = 198), if you write the correct R that is an integer not a fractional number and add a 4-th coulum with the current 1.6 reduction you will see is _almost_ very similar.

BTW don't tile the 4 columns otherwise is almost unreadable