Adding Transposition caused more nodes to be searched?

Discussion of chess software programming and technical issues.

Moderator: Ras

aschepis

Adding Transposition caused more nodes to be searched?

Post by aschepis »

I recently resurrected the chess engine i started in college and once i sped it up a bit and got it playing decently i decided to add a transposition hash to take it to the next level. I have this working now but it appears to have caused even more nodes to be searched per ply that w/o the transposition. I'm trying to figure out what could be causing this.

Bad move ordering? Wouldnt this also be the case in non-TT scenarios?

I'm using NegaScout for the search so perhaps i'm causing more re-searches w/ the TT, but i'm not sure why it would only happen in the case of transposition tables.

Also, it could be a bug, of course :) Any comments on debugging and testing techniques would be appreciated.
User avatar
hgm
Posts: 28356
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Adding Transposition caused more nodes to be searched?

Post by hgm »

It sounds fishy. Perhaps you should first collect some statistics on TT hits without using them to do anything: just count how many hits you have, how many of this have sufficient depth, how many of those with insufficient depth have a hash move, and how often this hash move is the move the search would have picked anyway. That might give you an idea what the problem is.
aschepis

Re: Adding Transposition caused more nodes to be searched?

Post by aschepis »

Thanks, i'll give that a try.

as an aside, i was using a triangular pv array to track my pv before i put the hashing in. now i track my pv using the hash nodes themselves, so the test your suggesting will require a bit more of a change (back to tracking pv w/ the array)

is this how the pv is normally tracked w/ TT or have i gone down the wrong path?
User avatar
hgm
Posts: 28356
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Adding Transposition caused more nodes to be searched?

Post by hgm »

My engines trac the PV through the TT, but how well that works might heavily depend on the replacement scheme (i.e. are entries close to the root sufficiently potected).
AndrewShort

Re: Adding Transposition caused more nodes to be searched?

Post by AndrewShort »

aschepis wrote:Thanks, i'll give that a try.

as an aside, i was using a triangular pv array to track my pv before i put the hashing in. now i track my pv using the hash nodes themselves, so the test your suggesting will require a bit more of a change (back to tracking pv w/ the array)

is this how the pv is normally tracked w/ TT or have i gone down the wrong path?
There apparently are engines that do that, but it's risky because as recursion works it's way up the tree, hash nodes can get replaced. The end result is that the final PV is NOT the PV associated with the final backed up score, which can cause you debugging headaches...
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Adding Transposition caused more nodes to be searched?

Post by bob »

aschepis wrote:I recently resurrected the chess engine i started in college and once i sped it up a bit and got it playing decently i decided to add a transposition hash to take it to the next level. I have this working now but it appears to have caused even more nodes to be searched per ply that w/o the transposition. I'm trying to figure out what could be causing this.

Bad move ordering? Wouldnt this also be the case in non-TT scenarios?

I'm using NegaScout for the search so perhaps i'm causing more re-searches w/ the TT, but i'm not sure why it would only happen in the case of transposition tables.

Also, it could be a bug, of course :) Any comments on debugging and testing techniques would be appreciated.
First, is this for a single position for for several? Sometimes the hash table gives you a more accurate score for a given depth, which can make the tree bigger. Best example is fine 70, where you have to search to depth (either 23 or 26, suddenly I don't remember which) to see winning the pawn, but you can solve it at depth 18 or so with a working hash table. The depth 18 search with hashing is usually bigger than the depth 18 search without, but you get the right answer and in much shorter time...
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Fine #70

Post by sje »

Fine #70:
[d] 8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Adding Transposition caused more nodes to be searched?

Post by sje »

bob wrote:Best example is fine 70, where you have to search to depth (either 23 or 26, suddenly I don't remember which) to see winning the pawn, but you can solve it at depth 18 or so with a working hash table. The depth 18 search with hashing is usually bigger than the depth 18 search without, but you get the right answer and in much shorter time...
With a transposition table, the winning move shows up at depth 18. However, the score bump doesn't occur until around depth 23. The first 29 iterations take less than one second total time.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Adding Transposition caused more nodes to be searched?

Post by bob »

sje wrote:
bob wrote:Best example is fine 70, where you have to search to depth (either 23 or 26, suddenly I don't remember which) to see winning the pawn, but you can solve it at depth 18 or so with a working hash table. The depth 18 search with hashing is usually bigger than the depth 18 search without, but you get the right answer and in much shorter time...
With a transposition table, the winning move shows up at depth 18. However, the score bump doesn't occur until around depth 23. The first 29 iterations take less than one second total time.
My score goes up at something less than 23, depending on settings. If your move ordering is _really_ good, you will see the win at depth=26. If your move ordering is worse, you will see it quicker. I have had versions that spotted it anywhere from 18-26... Reductions (search reductions) might postpone this a ply or two since it is an unsafe form of pruning, of course. I just tried and the current version finds Kb1 becomes best at depth=26, and the score goes from +1 to +4. Takes .05 seconds on a slow machine to get there however... Last time I ran it it was solved at depth=18...
Dek

Re: Adding Transposition caused more nodes to be searched?

Post by Dek »

sje wrote:
bob wrote:Best example is fine 70, where you have to search to depth (either 23 or 26, suddenly I don't remember which) to see winning the pawn, but you can solve it at depth 18 or so with a working hash table. The depth 18 search with hashing is usually bigger than the depth 18 search without, but you get the right answer and in much shorter time...
With a transposition table, the winning move shows up at depth 18. However, the score bump doesn't occur until around depth 23. The first 29 iterations take less than one second total time.
Hi,
Two questions, I m right now debugging my transposition table and I can't seem to find the material win even by searching to move 30+ . Could you post the PV to win material ? Also is there Fine 1-69 or 71... anywhere ? Where exactly are those from ?

Thanks for your reply.