How often do engines clear their hash memory

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Dann Corbit
Posts: 11112
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: How often do engines clear their hash memory

Post by Dann Corbit » Fri Apr 14, 2017 7:19 am

That's just it.

I have a 64 core machine, that I paid very little for (I actually guess it was a mistake since I see identically configured machines going for 20 times as much as I paid, but that is neither here nor there with the exponential decay of the cost of compute power).

In ten years, we will have giant piles of cores (since it is such a simple way to expand horsepower).

So ways to exploit the parallel nature of chess with a giant pile of cores may require new thinking.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

corres
Posts: 2873
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: How often do engines clear their hash memory

Post by corres » Fri Apr 14, 2017 9:26 am

[quote="syzygy"]
Stockfish does not clear its hash between moves. And it would be foolish to do so.
[/quote]

In fact, "clear hash" only works if you klick on to the "Newgame" button of a chess GUI.
This strange behavior may modifies the result of an automated engine-engine competition based on consecutive games.
To clear hash table causes waste of time for engine using this tool, I think this is the main reason of its disregarding.
But, if you analyse a game forth and back "clear hash" is needful tool.

corres
Posts: 2873
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: How often do engines clear their hash memory

Post by corres » Sat Apr 15, 2017 5:46 am

[quote="Dann Corbit"]
[quote="hgm"]
{snip}
The only reason I can see for clearing the table before every move is reproducibility of bugs; on a fully debugged engine there is no need to do that. Even clearing before a new game isn't really useful. (Unless you changed variant...)
{snip}
[/quote]

I guess that he is talking about analyzing a set of EPD positions (or Analyzing a game, rather than game play).

I have seen engines clear their hash between EPD rows/game positions when analyzing.

Much better to see if the position is related first rather than to clear unconditionally.
[/quote]

If chess engine has an option to clear hash, to clear or not to clear it is determined by the chess GUI.
I mainly use Fritz GUI and running in it Stockfish clear hash tool works in the case of manual using only.
There are a viewpoints for engine makers and there are other viewpoints for users of chess engines....

syzygy
Posts: 4554
Joined: Tue Feb 28, 2012 10:56 pm

Re: How often do engines clear their hash memory

Post by syzygy » Sat Apr 15, 2017 10:45 am

hgm wrote:
kranium wrote:This whole discussion reminds me just how inefficient the current UCI and WB 'ponder' specifications are.
Basically you're telling the engine to analyze the position after the opponent has made 1 specific move which you have to 'guess' (often/usually when there are many decent 'playable' candidate moves available).
As Robert Pope already mentioned: there is no such specification for WB protocol.
And the UCI protocol explicitly specifies that the engine is free to do whatever it wants. The pondermove thing is just there to simplify the implementation of the standard way of pondering (which anyway seems to be the most efficient method for the reasons you give).
* ponder
start searching in pondering mode.
Do not exit the search in ponder mode, even if it's mate!
This means that the last move sent in in the position string is the ponder move.
The engine can do what it wants to do, but after a "ponderhit" command it should execute the suggested move to ponder on. This means that the ponder move sent by the GUI can be interpreted as a recommendation about which move to ponder. However, if the engine decides to ponder on a different move, it should not display any mainlines as they are likely to be misinterpreted by the GUI because the GUI expects the engine to ponder on the suggested move.

User avatar
yurikvelo
Posts: 547
Joined: Sat Dec 06, 2014 12:53 pm

Re: How often do engines clear their hash memory

Post by yurikvelo » Fri Apr 17, 2020 9:40 pm

corres wrote:
Fri Apr 14, 2017 9:26 am
syzygy wrote: Stockfish does not clear its hash between moves. And it would be foolish to do so.
In fact, "clear hash" only works if you klick on to the "Newgame" button of a chess GUI.
This strange behavior may modifies the result of an automated engine-engine competition based on consecutive games.
To clear hash table causes waste of time for engine using this tool, I think this is the main reason of its disregarding.
But, if you analyse a game forth and back "clear hash" is needful tool.
Anybody, please clarify what is actually going on:

1) arena 3.5 Build 2816 GUI (Windows)
2) SF-dev (abrok)
3) Infinite analyze mode (hash % usage slowly grows and can reach up to 99%), multiPV=1
4) I manually play best move, suggested by SF

Immediately Hash = 0%

Is it expected behaviour? SF start search from scratch?

chrisw
Posts: 3188
Joined: Tue Apr 03, 2012 2:28 pm

Re: How often do engines clear their hash memory

Post by chrisw » Fri Apr 17, 2020 10:03 pm

yurikvelo wrote:
Fri Apr 17, 2020 9:40 pm
corres wrote:
Fri Apr 14, 2017 9:26 am
syzygy wrote: Stockfish does not clear its hash between moves. And it would be foolish to do so.
In fact, "clear hash" only works if you klick on to the "Newgame" button of a chess GUI.
This strange behavior may modifies the result of an automated engine-engine competition based on consecutive games.
To clear hash table causes waste of time for engine using this tool, I think this is the main reason of its disregarding.
But, if you analyse a game forth and back "clear hash" is needful tool.
Anybody, please clarify what is actually going on:

1) arena 3.5 Build 2816 GUI (Windows)
2) SF-dev (abrok)
3) Infinite analyze mode (hash % usage slowly grows and can reach up to 99%), multiPV=1
4) I manually play best move, suggested by SF

Immediately Hash = 0%

Is it expected behaviour? SF start search from scratch?
if that action advanced generation8 then the scan of 1000 hash entries (the way SF calculates hashfull) will declare that all those entries are available (0% hash use) even though the hash will still be full of best move entries that can still be read. Generation8 effectively just declares the hash entries from the prior search to be free now, capable to be written into.
It would restart from scratch *but* with hash table full of useful
Info.

User avatar
yurikvelo
Posts: 547
Joined: Sat Dec 06, 2014 12:53 pm

Re: How often do engines clear their hash memory

Post by yurikvelo » Sat May 23, 2020 9:24 pm

chrisw wrote:
Fri Apr 17, 2020 10:03 pm
It would restart from scratch *but* with hash table full of useful
Arena send move commands during infinite analysis this way:

Code: Select all

-->1:stop
-->1:position fen **************** moves **** **** **** **** **** my_new_move1
-->1:go infinite
...........
-->1:stop
-->1:position fen **************** moves **** **** **** **** **** **** my_new_move2
-->1:go infinite
is this fine with Hash?

Post Reply