about hash tables and illogical behaviour of chess programs

[zamar]

I have read the code actually, and I did it long before that post above. It is possible, though, that I did not do it carefully enough at the moment I wrote the above. The point is, as I wrote, I based my comment on the statement of Joona who has probably read the code, too

I wrote it, if I remember correctly But I do not always understand things I write

In fact, after looking into the code a second time, I still believe that Joona is right. The H.gain() function does *not* return just the difference between two consecutive evals, it is based on the gains table which collects some kind of maximum eval delta over time.

Yes, but the value from gain table is not pure eval delta maximum, that's why need the "correction".
const int ApproximateGainTableUncertaintyMargin = 45;

You may be right in stating that putting that "45" constant into the code that manages the gains table could reduce its readabilty, so I accept that argument against it, of course.

Marco is the professional developer, so I tend to trust on his judgements.

[Ralph Stoesser]

Or just add Joona's comment to the source code.
Btw. @Joona: Thank you for the clarification.


I have another question about the new search code.

Have you tried to use qsearch() instead of evaluate() as the basic node value estimate

I've thought a lot about it, but never tried. In fact I've spend a lot of time to planning "depth-R" scout search for pruning decisions in higher depths (which is just a generalization of what you are proposing).

The problem is of course that it's very expensive, but another great problem is those scout searches (also pure qsearch!) can have nasty side effects to transposition table, so it's again one of those things which (I believe) has a lot of potential, but which are irritatingly hard to get right.

I gave it a very short try with Stockfish 1.7, just quick&dirty replaces for single thread use, no other changes. The search was still very fast, but I've got serious instability problems after a few dozen moves. The search went mad somehow, the iterative deepening seemed to got broken, but no program crash.

I also have tried to use qsearch() within the MovePicker class for sorting "good" captures (SEE >= 0), also with no success, because of hard program crashes this time.

Because of time pressure I was not able to debug something.
After your encouraging reply I consider to start a debug session on weekend.
At least I could learn something about the internals ...

[zamar]

I gave it a very short try with Stockfish 1.7, just quick&dirty replaces for single thread use, no other changes. The search was still very fast, but I've got serious instability problems after a few dozen moves. The search went mad somehow, the iterative deepening seemed to got broken, but no program crash.

I also have tried to use qsearch() within the MovePicker class for sorting "good" captures (SEE >= 0), also with no success, because of hard program crashes this time.

Because of time pressure I was not able to debug something.
After your encouraging reply I consider to start a debug session on weekend.
At least I could learn something about the internals ...

If you succeed in getting sth out of it, let us know

[Ralph Stoesser]

Of course an immediate patch could be something along the line of:

Code: Select all

bool accurate = tte && (depth - tte->depth() < 3 * OnePly);

refinedValue < beta - razor_margin(depth) + (accurate ? extraRazorMargin : 0);

refinedValue >= beta + futility_margin(depth, 0)) - (accurate ? extraFutilityMargin : 0);

...but this needs test and is not very nice, at least to my eyes

+151, -128, =418
after 698 self play games at 1 min/game. 256MB hash, single thread.

[zamar]

+151, -128, =418
after 698 self play games at 1 min/game. 256MB hash, single thread.

Looks promising, but please continue until 1000 games (it's our minimum).
If the change still looks promising, please post _exact_ patch what did you test.

[ernest]

+151, -128, =418

so far:
51.65% or +11.55 Elo
standard deviation +- 8.4 Elo

[Ralph Stoesser]

+151, -128, =418

so far:
51.65% or +11.55 Elo
standard deviation +- 8.4 Elo

I'm still testing. After 1000 games the test version was 51,2% ahead.

That's what I try (regarding step 6+7 from search()):
refinedValue belong from evaluate(): less agressive pruning/razoring than default
refinedValue belong from shallow search: default pruning/razoring
refinedValue belong from deep search: more aggressive pruning/razoring

Right now I test a more aggressive scheme. I raise the razorDepth substantially in case refinedValue belong from deep search. Looks better, 53,7% after 752 games (30 sec/game), but not enogh games, I know... (and while I write this the default version makes up ground )

[mcostalba]

Looks better, 53,7% after 752 games (30 sec/game), but not enogh games, I know... (and while I write this the default version makes up ground )

Just an hint: razoring and pruning are very sensible to time control. It is known that at super blitz times aggressive pruning pays off but you cannot keep the same results at more moderate time control.

Another hint: (I know I am annoying ) do not mix ideas in a single test. I would suggest to stick with one scheme changing only the margin values until you find a good result. Then change something else.

Finally third and last hint: 51% is not bad. Big ELO increases are mad by little small steps, not by an handful of super features. So if you have a patch that scores 51% at 1 minute time control then please post it, I will test again and verify the result. Nothing prevents you to continue tweaking the formula but starting from an higher base, this is how normally we reach satisfactory results in a reliable way.

[Ralph Stoesser]

Hi Marco,

Yes, I kind of mix up things, especially in my second test, since I raise razorDepth, but for now I just like to play around a bit to get a feeling for Stockfish's search and also for testing changes.

From what I've seen in only roughly 10% of the cases refinedValue belong from search. So in 90% of the cases I actually lower the default magins for razoring and null move pruning, also in my first test.

I've seen major performance fluctuations in these 1000 self play games, escpecially in the last 300 games, though the test version was always ahead. I really don't trust these 1000 games. I have the impression that I definitely need more testing. And that's what I do right now, with two sweating computers.

[mcostalba]

Hi Marco,

Yes, I kind of mix up things, especially in my second test, since I raise razorDepth, but for now I just like to play around a bit to get a feeling for Stockfish's search and also for testing changes.

From what I've seen in only roughly 10% of the cases refinedValue belong from search. So in 90% of the cases I actually lower the default magins for razoring and null move pruning, also in my first test.

I've seen major performance fluctuations in these 1000 self play games, escpecially in the last 300 games, though the test version was always ahead. I really don't trust these 1000 games. I have the impression that I definitely need more testing. And that's what I do right now, with two sweating computers.

Hi Ralph,

yes results can fluctuate a lot, especially with very short time controls, with longer time controls you have less fluctuations.

Anyhow, finger corssed and I hope you come out with some interesting result....

BTW could be that, under your test conditions, lower the default magins for pruning and razoring gives better results _independently_ from the fact that you do it only for refined values. IOW perhaps lowering the margins without any further consideration could give even a better result, this would mean that the good result of refined value dependent pruning is just an artifact of lowering the margins in absolute terms.