Pawn hash table, a little disappointment

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
xr_a_y
Posts: 768
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Pawn hash table, a little disappointment

Post by xr_a_y » Mon Oct 28, 2019 6:05 am

I added a pawn hash table to Minic in order to store everything pawn related (passed, isolated, doubled, backward, candidate, holes, attack, openfile, semiopenfile, ...)

So the entry of the hash table is quite big (sizeof(PawnEntry) = 140) ...

I'm using a total size for this table that is 1/16 of the TT.
The table is well used, often there is a ratio of 10 read for only 1 write.

But this is a slowdown of about 10% in knps !

I wonder how others engines behave ?

Joost Buijs
Posts: 987
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Pawn hash table, a little disappointment

Post by Joost Buijs » Mon Oct 28, 2019 3:54 pm

xr_a_y wrote:
Mon Oct 28, 2019 6:05 am
I added a pawn hash table to Minic in order to store everything pawn related (passed, isolated, doubled, backward, candidate, holes, attack, openfile, semiopenfile, ...)

So the entry of the hash table is quite big (sizeof(PawnEntry) = 140) ...

I'm using a total size for this table that is 1/16 of the TT.
The table is well used, often there is a ratio of 10 read for only 1 write.

But this is a slowdown of about 10% in knps !

I wonder how others engines behave ?
In my engine the behavior is exactly the opposite, I get a speedup of 10% with just a small pawn hash-table of 8 MB per thread. I only store the evaluation score and passed pawn bitboards for both colors, nothing else.

RubiChess
Posts: 114
Joined: Fri Mar 30, 2018 5:20 am

Re: Pawn hash table, a little disappointment

Post by RubiChess » Mon Oct 28, 2019 4:27 pm

xr_a_y wrote:
Mon Oct 28, 2019 6:05 am
I'm using a total size for this table that is 1/16 of the TT.
The table is well used, often there is a ratio of 10 read for only 1 write.

But this is a slowdown of about 10% in knps !

I wonder how others engines behave ?
There must be something wrong with your code. I would have a look but it seems that it isn't in your github?
I remember that I got a visible speed up when I introduced pawn hash. My entry is 96 bytes.

User avatar
hgm
Posts: 23725
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Pawn hash table, a little disappointment

Post by hgm » Mon Oct 28, 2019 6:31 pm

How large is your table in MB? Making it so large that it cannot be easily contained in the L3 cache could be very counter-productive. Because most accesses would be very much slower, while the extra size would hardly give you any extra hits. Because even the small table might have a hit rate of 95% or more.

User avatar
xr_a_y
Posts: 768
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Pawn hash table, a little disappointment

Post by xr_a_y » Mon Oct 28, 2019 6:41 pm

RubiChess wrote:
Mon Oct 28, 2019 4:27 pm
xr_a_y wrote:
Mon Oct 28, 2019 6:05 am
I'm using a total size for this table that is 1/16 of the TT.
The table is well used, often there is a ratio of 10 read for only 1 write.

But this is a slowdown of about 10% in knps !

I wonder how others engines behave ?
There must be something wrong with your code. I would have a look but it seems that it isn't in your github?
I remember that I got a visible speed up when I introduced pawn hash. My entry is 96 bytes.
I'll push it soon. I'll try to store some part of it as score instead of bitboard when possible (isolated, doubled, candidate, backward) to have smaller entry.

User avatar
xr_a_y
Posts: 768
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Pawn hash table, a little disappointment

Post by xr_a_y » Mon Oct 28, 2019 6:43 pm

hgm wrote:
Mon Oct 28, 2019 6:31 pm
How large is your table in MB? Making it so large that it cannot be easily contained in the L3 cache could be very counter-productive. Because most accesses would be very much slower, while the extra size would hardly give you any extra hits. Because even the small table might have a hit rate of 95% or more.
Indeed larger than L3 cache right now, around 17Mb (my i9 has 16Mb cache). I'll try a smaller one.

User avatar
xr_a_y
Posts: 768
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Pawn hash table, a little disappointment

Post by xr_a_y » Mon Oct 28, 2019 6:48 pm

xr_a_y wrote:
Mon Oct 28, 2019 6:43 pm
hgm wrote:
Mon Oct 28, 2019 6:31 pm
How large is your table in MB? Making it so large that it cannot be easily contained in the L3 cache could be very counter-productive. Because most accesses would be very much slower, while the extra size would hardly give you any extra hits. Because even the small table might have a hit rate of 95% or more.
Indeed larger than L3 cache right now, around 17Mb (my i9 has 16Mb cache). I'll try a smaller one.
Using a 4Mb Pawn TT leads to this number of read/write on a middle game position. Which is less than 5 read per write. Isn't it too low ?

Code: Select all

ttPawnhits 1476819
ttPawnInsert 329855

User avatar
hgm
Posts: 23725
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Pawn hash table, a little disappointment

Post by hgm » Mon Oct 28, 2019 7:31 pm

In KingSlayer I use a 1MB table, but the entry size is only 32 bytes. So in the end I have more entries than your 4MB. But not much more. An 83% hit rate seems a bit low.

But even then, you should have reduced the time spent in executing Pawn-evaluation code by a factor 6. The only cost is the update of the Pawn key plus the lookup. It would be really strange if that would take more time (when the lookup comes rom cache) than the Pawn evaluation.

PK
Posts: 826
Joined: Mon Jan 15, 2007 10:23 am
Location: Warsza
Contact:

Re: Pawn hash table, a little disappointment

Post by PK » Mon Oct 28, 2019 7:46 pm

What is your pawn hash? Pawns only or pawns and kings? With pawns only, my speed gain has been very modest. With pawns and kings, hashing pawn shield and pawn storm scores, it became something like 10%.

User avatar
xr_a_y
Posts: 768
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Pawn hash table, a little disappointment

Post by xr_a_y » Mon Oct 28, 2019 8:19 pm

PK wrote:
Mon Oct 28, 2019 7:46 pm
What is your pawn hash? Pawns only or pawns and kings? With pawns only, my speed gain has been very modest. With pawns and kings, hashing pawn shield and pawn storm scores, it became something like 10%.
Pawn + King hash but not really storing King things for now

Post Reply