Pawn Hash

[hgm]

Perhaps it would not be so detrimental to the hit rate if you only partially resolve the King position. In the middle game there are a lot of squares that could be considered "universally bad". E.g. it does not matter very much if a white King is on a8, d8 or g6 in the middle game. If there is any difference, the PST can take care of that.

What I would basically want is a Pawn evaluation that depends on if my King is near g1 or c1. If it is on g1, having no Pawns on the g and h file is not so hot, while with the King on c1 this might be perfectly acceptable. And I don't care so much if the King is on g1, h1 or g2. That is not a strategic feature, and will be solved in a single move. So presumably the search would solve it, and if not, scoring them wildly different would only invite horizon effect.

So in practice the board for the white King can be divided into 3 zones (a-c1-3, f-h1-3 and the rest), each having their own Zobrist key.

[Gerd Isenberg]

Perhaps it would not be so detrimental to the hit rate if you only partially resolve the King position. In the middle game there are a lot of squares that could be considered "universally bad". E.g. it does not matter very much if a white King is on a8, d8 or g6 in the middle game. If there is any difference, the PST can take care of that.

What I would basically want is a Pawn evaluation that depends on if my King is near g1 or c1. If it is on g1, having no Pawns on the g and h file is not so hot, while with the King on c1 this might be perfectly acceptable. And I don't care so much if the King is on g1, h1 or g2. That is not a strategic feature, and will be solved in a single move. So presumably the search would solve it, and if not, scoring them wildly different would only invite horizon effect.

So in practice the board for the white King can be divided into 3 zones (a-c1-3, f-h1-3 and the rest), each having their own Zobrist key.

The usual way is to calculate and hash pawn-shield stuff for all relevant king areas simultaneously, to later apply the relevant stuff. Since the hit-rate is so high, wasting some time for not relevant stuff doesn't matter.

[hgm]

That is possible, but it is also not free in terms of storage. Each entry would need to store scores for the Pawn structiure with the King in three different zones, plus probably an end-game score. And you would have to do it for both the white and black King, 9 combinations (if transition to the end-game is considered a global thing). If you would hash depending on zone and game stage, you would only need to store one score per entry. Of course you would need to store a signature for each of those, so if they would all be in the table, that would require 10 signatures, which almost certainly takes much more space than 10 scores.

But of course they won't all be in the table. I white castled Q-side, and black K-side, there will be virtually no positions that have that differently in the tree. With all the scores in a single entry, it would only mean you have 9 unused score slots.

[mathmoi]

Including also pawn shield calculation into the pawn hash would IMO require to include the king positions into the pawn hash key, too. Don't know if that makes sense, I have never tried that.

I started using this recently and it was a good improvement. I do not xor the king location in to the zobrist key though. It reduces the hit rate from 97% to 50 something IIRC. I just store the two king locations along with other data in the pawn hash entry. So with this i calculate pawn scores which depend on king locations such as pawn shield, pawn storm, penalty for open files around king and bad king location by itself.
May be xor-ing king locations may be a good idea in king-pawn endings but i doubt it matters much.

Hi Daniel,

Are you telling that you don't include the positions of the kings in the hash key, but you store informations/evaluation that are dependent on the kings position in the hash entry? How can that work?

It sounds like he also stores the kings positions in the entry, and only uses the cached pawn shield, pawn storm info if the king is still in the same position. I guess until you get to the endgame, your king is probably hiding behind some pawns and neither the king nor the pawns is very active, so you'll still be able to re-use this information in a lot of nodes.

Ok, if he check for the position of the kings before using any king dependent data, it makes sense.

[Daniel Shawul]

Yes, i check whether any of the two kings moved. This gives me good hit rates. Also the calculation is avoided when king safety calculations are not done, so you need to stop doing this once queens are exchanged f.i
cheers
Daniel

[Daniel Shawul]

Zoning do not work for me because i calculate more than pawn shield which depend on specific king locations. My open files around king eval, and pawn storm depend on the exact location of the king.

[Dann Corbit]

Zoning do not work for me because i calculate more than pawn shield which depend on specific king locations. My open files around king eval, and pawn storm depend on the exact location of the king.

Many programs include the king position in the pawn hash.

[bob]

That is possible, but it is also not free in terms of storage. Each entry would need to store scores for the Pawn structiure with the King in three different zones, plus probably an end-game score. And you would have to do it for both the white and black King, 9 combinations (if transition to the end-game is considered a global thing). If you would hash depending on zone and game stage, you would only need to store one score per entry. Of course you would need to store a signature for each of those, so if they would all be in the table, that would require 10 signatures, which almost certainly takes much more space than 10 scores.

But of course they won't all be in the table. I white castled Q-side, and black K-side, there will be virtually no positions that have that differently in the tree. With all the scores in a single entry, it would only mean you have 9 unused score slots.

Use some sort of "defect counter". 0 = optimal pawns (unmoved), as the number climbs, the king is less safe. You can store that in one byte. So one byte for king safety shelter on queen-side, one for kingside, one for king in center. 3 bytes, all hashed, no eval overhead. Just use the one that matches where the king currently is located.

[bob]

Zoning do not work for me because i calculate more than pawn shield which depend on specific king locations. My open files around king eval, and pawn storm depend on the exact location of the king.

Many programs include the king position in the pawn hash.

I hope not, it absolutely kills hash hit rate. In Cray Blitz we actually did a king safety hash, which included pawns + K in a separate table. The hit rate was never even 50%, whereas the normal pawn hash hit rate (for me) is always > 99%. most pawn scoring has nothing to do with king position. Giving up 50% hits could make a program many times slower than it needs to be.

[hgm]

I wonder if including Bishop color in the Pawn hash key would be a good idea. This would only require two extra Zobrist keys per side, one for the white squares, and one for the black squares, and the status would be very stable through the tree. So it might not degrade the hit-rate too much.

You could then put the good Bishop / bad Bishop distinction in the hashed pawn scores. You could also reduce the value of an extra pawn when there are unlike Bishops.