Pawnhash problems

Discussion of chess software programming and technical issues.

Moderator: Ras

EricLang

Pawnhash problems

Post by EricLang »

I use a pawnhash in my chessprogram and I'm running into problems.
I'm relatively new to hashing and wanted to ask a question.
I have this PawnHashElement-record:

Code: Select all

  TPawnHashElement = record
    Key            : U64;        // pawnhashkey
    CollisionCheck : U64;        // board.hashkey to doublecheck collisions
    Evaluation     : Integer;    // pawn evaluation
  end;
and the pawnhashtable has 65536 * 8 entries.
I noticed that with only one 64-bits number I got a lot of wrong evaluations from the table so I added an extra CollisionCheck 64-bits number and this seems to work well.

The question is: do I really need two 64-bits numbers to avoid retreiving a wrong evaluation from the hash-table? Is one not enough?

BTW: I use random ZobristKeys: for each square/piece a random number which is xor-ed when the board changes.

The code for storing:

Code: Select all

procedure TPawnHash.Store(const B: TBoard; aEvaluation: Integer);
begin
  with fArray^[B.PawnHashKey mod Size] do
  begin
    Key := B.PawnHashKey;
    Evaluation := aEvaluation;
    CollisionCheck := B.HashKey;
  end;
end;
The code for probing:

Code: Select all

function TPawnHash.Probe(const B: TBoard): Integer;
begin
  with fArray^[B.PawnHashKey mod Size] do
  begin
    if (Key = B.PawnHashKey) and (CollisionCheck = B.HashKey) then
      Result := Evaluation
    else
      Result := PROBE_UNKNOWN;
  end;
end;
Edmund
Posts: 670
Joined: Mon Dec 03, 2007 3:01 pm
Location: Barcelona, Spain

Re: Pawnhash problems

Post by Edmund »

Firstly, the mod command is slow. Many programs use a size that is a power of 2 and then get the index by anding with size-1

Usually the pawn transposition table is used for only storing the perception of the pawn-structure. No values about other pieces on the board influence this value. Possible parameters would for example be evaluating passers, weak pawns, etc. So make sure that you don't consider values like rook on open file etc. this has to be calculated separately.

maybe there is a problem with the pawnhashkey. Make sure you only change it in case of a pawnmove/pawn gets captured/pawn promotes/undo promotion. Probably you won't even need to xor the stm key after each move, unless you have any side depended evaluation terms.

using the board.hashkey as a second check, doesn't make sense. The idea of the pawnhash-table is to have a small table which hold all pawnformations. Thus the hitrate is quite high (>90%). If you now use the global hashkey as well, the hitrate goes down.

So I don't know what you mean by a collision, but a collision where all the pawns are on the same positions is actually wanted.
EricLang

Re: Pawnhash problems

Post by EricLang »

Hmm, I think I solved the problem. I did not store the GameStage of the evaluation. Different GameStages give different evaluations :)
EricLang

Re: Pawnhash problems

Post by EricLang »

Thanks for the answer. The pawnhashkey is always correct I think.
And I do indeed only store the pawn-evaluation in the pawnhash.
I'll check if the mod command is really slow. I didn't know that.

(Maybe my "collision" terminoligy was wrong. What I meant was that I thought I got wrong positions from the hash, but as I said I think I solved the problem).
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Pawnhash problems

Post by bob »

EricLang wrote:I use a pawnhash in my chessprogram and I'm running into problems.
I'm relatively new to hashing and wanted to ask a question.
I have this PawnHashElement-record:

Code: Select all

  TPawnHashElement = record
    Key            : U64;        // pawnhashkey
    CollisionCheck : U64;        // board.hashkey to doublecheck collisions
    Evaluation     : Integer;    // pawn evaluation
  end;
and the pawnhashtable has 65536 * 8 entries.
I noticed that with only one 64-bits number I got a lot of wrong evaluations from the table so I added an extra CollisionCheck 64-bits number and this seems to work well.
This should not happen. There are not that many possible pawn positions in the game of chess. If you are getting bogus matches, I'd look at the hashing algorithm first and make sure that you are really using 64 bit random numbers and not screwing up the hash signature somewhere

The question is: do I really need two 64-bits numbers to avoid retreiving a wrong evaluation from the hash-table? Is one not enough?
one 64 bit zobrist key is enough, just like we use for normal hashing. Why would 64 bits be good for positions with both pawns and pieces in the hash table, but with just pawns, suddenly you need _more_ bits???

BTW: I use random ZobristKeys: for each square/piece a random number which is xor-ed when the board changes.

The code for storing:

Code: Select all

procedure TPawnHash.Store(const B: TBoard; aEvaluation: Integer);
begin
  with fArray^[B.PawnHashKey mod Size] do
  begin
    Key := B.PawnHashKey;
    Evaluation := aEvaluation;
    CollisionCheck := B.HashKey;
  end;
end;
The code for probing:

Code: Select all

function TPawnHash.Probe(const B: TBoard): Integer;
begin
  with fArray^[B.PawnHashKey mod Size] do
  begin
    if (Key = B.PawnHashKey) and (CollisionCheck = B.HashKey) then
      Result := Evaluation
    else
      Result := PROBE_UNKNOWN;
  end;
end;
EricLang

Re: Pawnhash problems

Post by EricLang »

Actually this CollisionCheck code was complete nonsense and totally wrong, sorry for that.
As far as I can see the pawnhash works ok now.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Pawnhash problems

Post by mcostalba »

bob wrote: This should not happen. There are not that many possible pawn positions in the game of chess.
This is a bit off-topic, but I would like to propose this idea I got (and actually implemented).

I have found that a slow part of evaluation is to calculate king's pawns shelter value. My idea was to add to pawn hash also king position, so that I can calculate king pawns shelter once and store in the pawn hash togheter with the value.

Good: you do not calculate king shelter for each call to evaluate() anymore.

Bad: when king moves you need to add a new pawn hash, so pawn hash entries will be more and you'll get more calls to pawn structure evaluation.

I didn't got a clear result, tests are almost neutral, perhaps slightly better but under the error thresold.

Could anyone test this to see if it gives something good ?

Thanks
Marco
gladius
Posts: 568
Joined: Tue Dec 12, 2006 10:10 am
Full name: Gary Linscott

Re: Pawnhash problems

Post by gladius »

Using the king position in the hash is a possibility. Another possibility is to compute all the potential pawn safety scores for your king, and store them in the pawn hash table. If you only have a few scores (like queenside, center, kingside) this ends up having very low overhead. You keep the benefits of having a 90%+ pawn hash rate, and just use the appropriate king safety score from the hash depending on your king position.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Pawnhash problems

Post by mcostalba »

gladius wrote:Using the king position in the hash is a possibility. Another possibility is to compute all the potential pawn safety scores for your king, and store them in the pawn hash table. If you only have a few scores (like queenside, center, kingside) this ends up having very low overhead. You keep the benefits of having a 90%+ pawn hash rate, and just use the appropriate king safety score from the hash depending on your king position.
But the king is not tied to three positions only (queenside, center, kingside) he can moves everywhere. You don't know where the king will go before castling.

I should calculate the king safety score for at least 16 king positions, the first two ranks, and store this in pawn hash, but doing this each pawn hash is much more cumbersome to calculate.

There are also tricky parts where the king moves out/in his _zone_ should I drop the score? It seems tricky a value that suddendly appears/disappears.

Am I missing something?

Marco
Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Re: Pawnhash problems

Post by Edsel Apostol »

mcostalba wrote:
gladius wrote:Using the king position in the hash is a possibility. Another possibility is to compute all the potential pawn safety scores for your king, and store them in the pawn hash table. If you only have a few scores (like queenside, center, kingside) this ends up having very low overhead. You keep the benefits of having a 90%+ pawn hash rate, and just use the appropriate king safety score from the hash depending on your king position.
But the king is not tied to three positions only (queenside, center, kingside) he can moves everywhere. You don't know where the king will go before castling.

I should calculate the king safety score for at least 16 king positions, the first two ranks, and store this in pawn hash, but doing this each pawn hash is much more cumbersome to calculate.

There are also tricky parts where the king moves out/in his _zone_ should I drop the score? It seems tricky a value that suddendly appears/disappears.

Am I missing something?

Marco
For simplicities sake, in my engine I calculate pawn king shelter from three squares only. These are b1/b8, e1/e8, g1/g8. I store them in the pawn hash and score king safety accordingly based on the area/file of the board where it resides. a-c for queenside, d-f for center and f-h for kingside.

There seems to be no case of a value that suddenly appears/disappears. Anywhere the king is, there will always be a score.