In testsuites: is hash memory cleared between positions?

[Jouni]

Should it be cleared automatically or should GUI send clear hash / reload engine command? This has been unclear to me.

Jouni

[Dann Corbit]

Should it be cleared automatically or should GUI send clear hash / reload engine command? This has been unclear to me.

Jouni

My opinion is that if the position has a hash hit, don't clear it. If the position analyzed is not in the hash then clear it.

The reason is that if (for instance) you are analyzing the moves of a game in sequence, you will get a huge speedup from not clearing the hash. If (on the other hand) you are analyzing a test set where the positions do not have any relationships between each other (besides perhaps the theme) then you should clear it.

A test to see if the position is already stored in the hash table is fast.

[Dirt]

My opinion is that if the position has a hash hit, don't clear it. If the position analyzed is not in the hash then clear it.

So if you analyze a game at ply N and follow by analyzing ply N-1 you would clear the hash? That doesn't sound optimal.

[Houdini]

Should it be cleared automatically or should GUI send clear hash / reload engine command? This has been unclear to me.

Jouni

The GUI should send a "ucinewgame" command before every position, to indicate that the new position is unrelated to the previous.

Robert

[hgm]

The GUI should send a "ucinewgame" command before every position, to indicate that the new position is unrelated to the previous.

But that still would not force the implementation to clear the hash. As long as you stay in the same variant, hashed positions will remain valid, and could occur in the new game. That even holds for sub-variants like Chess960. Only ehen you switch to a truly new variant, such as Chinese Chess, it would be better to clear the hash, as any hit you might get will surely be a key collission.

But with a strong key you wouldn't even have to worry about that, because in practice it will never happen. Assuming the engine is smart enough to pick another starting key in another variant. Fairy-Max is not smart enough to do that, so there I do clear the hash table for every variant switch.

[Teemu Pudas]

As long as you stay in the same variant, hashed positions will remain valid, and could occur in the new game.

Chess960 positions with subtly different castling rights will become invalid unless your castling key depends on the rook squares (which is a good idea).

[rbarreira]

In testsuites you would definitely want the performance to be independent between the different test cases, so one way or the other the hash should be cleared.

[hgm]

I think that both for WB and UCI protocol, the only sure way to clear the hash is to restart the egine.

Some egines has a private option to clear the hash, but it is not stadardized, so it is difficult for a GUI to use it.

[Mike S.]

I don't know how GUIs handle this. Often, when using not too big testsuites, I have even cleared the hash manually, between the positions each.

But now I tend to think it is not required, because normally, the positions in typical test suites are not at all related to each other. So, a "normal" influence from old hash entries seems impossible, with the rare exception of hash collisions I guess.

But I am only a user, not a programmer... If there are no reasons against clearing hash between positions, e.g. in features like "automatic test suite processing", I would certainly not object. I wonder if very large hash like 512 MB or more can be cleared in very short time, each? It shouldn't slow down the test process e.g. if 1,000 positions are tested with 3 seconds each or the like.

The more important point to have clean "out of the box" test conditions (if desired) is probably to delete, or temporarily move away the permanent learning files some engines create. Some testers deactivate that feature even for practical test games. It's almost a philosophy question; I wouldn't do it. - But these are different topics.

[Dann Corbit]

I think that both for WB and UCI protocol, the only sure way to clear the hash is to restart the egine.

Some egines has a private option to clear the hash, but it is not stadardized, so it is difficult for a GUI to use it.

I would hope that there is an option to prevent that.
I freqently analyze game positions in sequence.