Testing Hash Table with perft

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Testing Hash Table with perft

Post by diep »

hgm wrote:He is not measuring performance, but correctness...
that's not reality of course, you also keep referring in postings here on what worked for perft in hashtable and what didn't. Would be bad if another guy is doing that.

Once test done, people keep referring to it.

It's like ROM. Gets written once, rest of life just gets read :)
Felpo

Re: Testing Hash Table with perft

Post by Felpo »

yes,
I'm just measuring the correctness: my performance now are reasonable ( in term of move generation ) for a C# program as mine: I can't beat the HG Mueller qperft, neither the "emptran" shown in the Steven Edwards post, but is competitive in term of speed with some other engines I found, even in C++.
This obsession for testing correctness derive from my previous engine: I was unexperienced, and I created a slow and buggy move generator, I had a TT as well, but who can trust without a solid testing fundation ? So I ended in avery poor job, in an untestable search and a very uncompetitive engine.
I did not find any other way to probe if TT is working correct than challenge it on a perft session.
This appear to be tricky as well, because a big testing with a lot of position, but ending at perft 6 was perfectly working, to discover errors I had to go down to perft 9, and I'm still not sure on what will happen at perft 10 :)
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Testing Hash Table with perft

Post by diep »

Felpo wrote:yes,
I'm just measuring the correctness: my performance now are reasonable ( in term of move generation ) for a C# program as mine: I can't beat the HG Mueller qperft, neither the "emptran" shown in the Steven Edwards post, but is competitive in term of speed with some other engines I found, even in C++.
This obsession for testing correctness derive from my previous engine: I was unexperienced, and I created a slow and buggy move generator, I had a TT as well, but who can trust without a solid testing fundation ? So I ended in avery poor job, in an untestable search and a very uncompetitive engine.
I did not find any other way to probe if TT is working correct than challenge it on a perft session.
This appear to be tricky as well, because a big testing with a lot of position, but ending at perft 6 was perfectly working, to discover errors I had to go down to perft 9, and I'm still not sure on what will happen at perft 10 :)
Maybe switch language, speed increase of factor 4 or so?

C is a lot faster than the object oriented languages practical spoken.

Vincent
Felpo

Re: Testing Hash Table with perft

Post by Felpo »

diep wrote:
Felpo wrote:yes,
I'm just measuring the correctness: my performance now are reasonable ( in term of move generation ) for a C# program as mine: I can't beat the HG Mueller qperft, neither the "emptran" shown in the Steven Edwards post, but is competitive in term of speed with some other engines I found, even in C++.
This obsession for testing correctness derive from my previous engine: I was unexperienced, and I created a slow and buggy move generator, I had a TT as well, but who can trust without a solid testing fundation ? So I ended in avery poor job, in an untestable search and a very uncompetitive engine.
I did not find any other way to probe if TT is working correct than challenge it on a perft session.
This appear to be tricky as well, because a big testing with a lot of position, but ending at perft 6 was perfectly working, to discover errors I had to go down to perft 9, and I'm still not sure on what will happen at perft 10 :)
Maybe switch language, speed increase of factor 4 or so?

C is a lot faster than the object oriented languages practical spoken.

Vincent
As I learned from previous thread over this forum, the move generation speed is not influencing too much the overall performance of the engine: Achieving more cut-off is for sure better than a 4*speed in the move generation process. And Actually the main topic in this particular thread is not really speed, but how to probe if the TT behave correctly ( even if as a side effect they improve the perft speed too ).
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Testing Hash Table with perft

Post by diep »

Felpo wrote:
diep wrote:
Felpo wrote:yes,
I'm just measuring the correctness: my performance now are reasonable ( in term of move generation ) for a C# program as mine: I can't beat the HG Mueller qperft, neither the "emptran" shown in the Steven Edwards post, but is competitive in term of speed with some other engines I found, even in C++.
This obsession for testing correctness derive from my previous engine: I was unexperienced, and I created a slow and buggy move generator, I had a TT as well, but who can trust without a solid testing fundation ? So I ended in avery poor job, in an untestable search and a very uncompetitive engine.
I did not find any other way to probe if TT is working correct than challenge it on a perft session.
This appear to be tricky as well, because a big testing with a lot of position, but ending at perft 6 was perfectly working, to discover errors I had to go down to perft 9, and I'm still not sure on what will happen at perft 10 :)
Maybe switch language, speed increase of factor 4 or so?

C is a lot faster than the object oriented languages practical spoken.

Vincent
As I learned from previous thread over this forum, the move generation speed is not influencing too much the overall performance of the engine: Achieving more cut-off is for sure better than a 4*speed in the move generation process. And Actually the main topic in this particular thread is not really speed, but how to probe if the TT behave correctly ( even if as a side effect they improve the perft speed too ).
A hashtable that keeps your search 100% intact might not be so efficient in an engine which goal is not to count how many moves each subtree has, yet focuses upon playing the best move instead :)

Additionally to that overwrite strategy in tic-tac-toe is different from chess engines.
Felpo

Re: Testing Hash Table with perft: Solved

Post by Felpo »

Well,
I finally done the full test. All works ok now. Problem was really a stupid bug during incremental ZKey update in castlings, showing wrong calc only at perft 9.
The test I ran perform a full Zobrist key recalc after each move/undomove and asserts the equality with the incemental one. No asserts in perft 9 , and exact move count: now I think I can trust my ZKey/hashtable.
Thanks to everybody for the suggestions.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Testing Hash Table with perft: Solved

Post by sje »

Note that you will have to test to depth ten or greater from the initial position to trigger black queenside castling.

I forget who first mentioned the idea, but another test strategy would be to take a well distributed problem suite like WAC (300 positions) and run perft N on each and sum the results.
Felpo

Re: Testing Hash Table with perft: Solved

Post by Felpo »

sje wrote:Note that you will have to test to depth ten or greater from the initial position to trigger black queenside castling.

I forget who first mentioned the idea, but another test strategy would be to take a well distributed problem suite like WAC (300 positions) and run perft N on each and sum the results.
I'm interested on submit more precalculated test to my move generation subsystem, I used perftestsuite.epd, that I found looking at Roce chess engine, It contains some insidious positions as this one:
r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1
and many others. Where can I found the WAC you are pointing me ?
Thanks
Felpo

Re: Testing Hash Table with perft: Solved

Post by Felpo »

Felpo wrote:
sje wrote:Note that you will have to test to depth ten or greater from the initial position to trigger black queenside castling.

I forget who first mentioned the idea, but another test strategy would be to take a well distributed problem suite like WAC (300 positions) and run perft N on each and sum the results.
I'm interested on submit more precalculated test to my move generation subsystem, I used perftestsuite.epd, that I found looking at Roce chess engine, It contains some insidious positions as this one:
r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1
and many others. Where can I found the WAC you are pointing me ?
Thanks
Never mind about where to find it, I suppose it is:
http://www.jakob.at/steffen/hossa/testsuites/wac.epd
Felpo

Re: Testing Hash Table with perft: Solved

Post by Felpo »

As a further reference, the file I'm currently using to do some automatic test:
http://cid-b8821720666a55e7.skydrive.li ... tsuite.epd
I'm not the author of this test suite, but I share because was very helpful to me in the first stage of move generator creation...