Looking for suggestions for perft test posititons

[rvida]

Has anyone done perft(n) for WAC or for similar test suites? It would be helpful to compare results.

WAC perft numbers up to depth 5 (generated by Critter)

Code: Select all

...

Interesting. Do you have the grand total sum for each depth value? (Just five numbers in this case.)

depth 1: 11962
depth 2: 393416
depth 3: 15784129
depth 4: 550761277
depth 5: 22347090274

[sje]

depth 1: 11962
depth 2: 393416
depth 3: 15784129
depth 4: 550761277
depth 5: 22347090274

Thank you for the data. I'll try to get a FEN file perft command running later today.

[sje]

depth 1: 11962
depth 2: 393416
depth 3: 15784129
depth 4: 550761277
depth 5: 22347090274

BozoChess now has a file perft command. But there are some different results! Only the counts for depth = {1, 2} match.

Code: Select all

[] ffperft WAC.fen 1
Count: 11962   Time: 000.00:00:00.045   Frequency: 265822
[] ffperft WAC.fen 2
Count: 393416   Time: 000.00:00:00.053   Frequency: 7422943
[] ffperft WAC.fen 3
Count: 15796091   Time: 000.00:00:00.950   Frequency: 16627464
[] ffperft WAC.fen 4
Count: 551178617   Time: 000.00:00:37.951   Frequency: 14523428

[sje]

Also different:

Code: Select all

[] ffperft WAC.fen 5
Count: 22363709083   Time: 000.00:22:01.410   Frequency: 16924125

[sje]

I have a set here of 6000+ positions with perft 1-6.
I generated the set by playing random games upto some randomized game length, and recorded the position every 3 half moves.
The counts have been reproduced by 3 different engines (all by me though).
The last engine perft bug I uncovered was in position 403. It was my goal to have a set that is large enough so that it would expose all new bugs within the first 10% and that the remaining 90% is there for confidence. (That means: if I would find a new bug in that 90% part and not earlier, the set would have to be increased).

For #403, Bozo says:

Code: Select all

[] perfttran 6
Depth: 6
FEN: rnb1kb1r/1p1p1ppp/p1p2n2/4p3/P5P1/1qB1P3/2PPNP1P/RN1QKB1R b KQkq - 0 1
Count: 1262436649   Time: 000.00:00:33.547   Frequency: 37631879

Which matches your result. To process your entire suite, it would help to have all the records on a single FEN (or EPD) file and then compare single number results by depth.

[marcelk]

I have a set here of 6000+ positions with perft 1-6.
I generated the set by playing random games upto some randomized game length, and recorded the position every 3 half moves.
The counts have been reproduced by 3 different engines (all by me though).
The last engine perft bug I uncovered was in position 403. It was my goal to have a set that is large enough so that it would expose all new bugs within the first 10% and that the remaining 90% is there for confidence. (That means: if I would find a new bug in that 90% part and not earlier, the set would have to be increased).

For #403, Bozo says:

Code: Select all

[] perfttran 6
Depth: 6
FEN: rnb1kb1r/1p1p1ppp/p1p2n2/4p3/P5P1/1qB1P3/2PPNP1P/RN1QKB1R b KQkq - 0 1
Count: 1262436649   Time: 000.00:00:33.547   Frequency: 37631879

Which matches your result. To process your entire suite, it would help to have all the records on a single FEN (or EPD) file and then compare single number results by depth.

I can recommend the 'awk' utility to prepare such data into any desired arrangement.

[sje]

Also different:

Code: Select all

[] ffperft WAC.fen 5
Count: 22363709083   Time: 000.00:22:01.410   Frequency: 16924125

Win at Chess to depth six:

Code: Select all

[] ffperft WAC.fen 6
Count: 809264359148   Time: 000.04:46:18.658   Frequency: 47108706

This will likely be the last of these until I get the EPD subsystem in place.

[Tom Likens]

My soon-to-be-released BozoChess program has a built-in self test routine which includes a set of perft test positions. Each position is sent to the perft bulk counter for depths of one to three ply and the results are checked against known values. Since the bulk counter can do move than ten million positions per second on my iMac, there is no noticeable time take during program initialization to run the self test.

But I have only a few positions and would like to add more, and so I'm looking for suggestions.

I have a set here of 6000+ positions with perft 1-6.
I generated the set by playing random games upto some randomized game length, and recorded the position every 3 half moves.
The counts have been reproduced by 3 different engines (all by me though).
The last engine perft bug I uncovered was in position 403. It was my goal to have a set that is large enough so that it would expose all new bugs within the first 10% and that the remaining 90% is there for confidence. (That means: if I would find a new bug in that 90% part and not earlier, the set would have to be increased).

I've recently started looking at Djinn again and as part of the neural reboot I decided to add some new interface commands. One of the classics that everyone else seems to have is the 'divide' command, which is used to debug perft problems. Interestingly, during this process I was able to fully use it because I ran across a castling bug , which I've since fixed. To really bang on the move generator I ran through all the perft test positions I could find online (at the wiki, etc.) plus all 6838! perft positions from Marcel's website and my numbers agree with his. If anyone is interested I converted all the positions to something a bit more digestable, so that the file now has the format:

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 ;D1 20 ;D2 400 ;D3 8902 ;D4 197281 ;D5 4865609 ;D6 119060324
rnbqkbnr/pp1ppppp/2p5/8/6P1/2P5/PP1PPP1P/RNBQKBNR b KQkq - 0 1 ;D1 21 ;D2 463 ;D3 11138 ;D4 274234 ;D5 7290026 ;D6 195464529
rnb1kbnr/ppq1pppp/2pp4/8/6P1/2P5/PP1PPPBP/RNBQK1NR w KQkq - 0 1 ;D1 27 ;D2 734 ;D3 20553 ;D4 579004 ;D5 16988496 ;D6 494216738

etc.

I can make it available at my web page if anyone is interested.

Here's the perft counts according to the perl script I hacked together two seconds ago.

D1 sum = 178,742
D2 sum = 4,689,446
D3 sum = 139,639,907
D4 sum = 4,135,458,706
D5 sum = 132,117,547,121
D6 sum = 4,185,882,506,133

Total = 4,322,280,020,055

Yes, that's over 4 TRILLION positions!

BTW, one thing I noticed when debugging this, is that no other program I tried orders the output of the 'divide' command by either the move name or even more useful, by the move counts. Obviously, this is solved with a simple sort, but it would be nice if that was done automatically. The move count is the better choice (IMO), because none of the programs put out a standard move format. Some produced short SAN, others long SAN, some from-to square algebraic etc. The move counts were (generally) the same.

regards,
--tom

[ankan]

when do you expect the first draft 12 record of Perft(13) count? In the case that the first draft 12 record is for 1.-a3, I hope that it will match with 54,239,338,583,061,004 (given by Paul Byrne six weeks ago)

Here are first 9 partial sums for perft(13) computed using my gpu perft program before it crashed (due to system instability or a bug )
https://docs.google.com/file/d/0B4d0eT2 ... sp=sharing

Code: Select all

0: ..a2-a3       54239338583061004
1: ..a2-a4       79097912720718078
2: ..b2-b3       72470867205869240
3: ..b2-b4       73447518515970566
4: ..c2-c3       85630662795901357
5: ..c2-c4       97580419694352415
6: ..d2-d3      141516447571471634
7: ..d2-d4      213234089428603755
8: ..e2-e3      241123823129738679

[ankan]

never mind - just realized that the thread is over 2 years old and perft(13) partial sums have already been calculated
http://chessprogramming.wikispaces.com/ ... +Positions