Looking for suggestions for perft test posititons

[sje]

My soon-to-be-released BozoChess program has a built-in self test routine which includes a set of perft test positions. Each position is sent to the perft bulk counter for depths of one to three ply and the results are checked against known values. Since the bulk counter can do move than ten million positions per second on my iMac, there is no noticeable time take during program initialization to run the self test.

But I have only a few positions and would like to add more, and so I'm looking for suggestions. The list so far:

Code: Select all

    fen := 'rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1';
    fen := '2qrr1n1/3b1kp1/2pBpn1p/1p2PP2/p2P4/1BP5/P3Q1PP/4RRK1 w - - 0 1';
    fen := 'r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1';
    fen := '8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1';
    fen := 'r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1';
    fen := 'r2q1rk1/pP1p2pp/Q4n2/bbp1p3/Np6/1B3NBn/pPPP1PPP/R3K2R b KQ - 0 1';
    fen := 'rnbqkb1r/ppppp1pp/7n/4Pp2/8/8/PPPP1PPP/RNBQKBNR w KQkq f6 0 3';

[Steve Maughan]

Hi Steven,

I'll try to come up with a few. Over the years I've found the following traps have tripped up my perft:

1. Double e.p. capture e.g. BP on a4 and c4, WP on b2
2. Castling gives check e.g. BK on f3, WK on e1 and WR on h1
3. Promotion of a pawn to a rook gives check (e.g. WP on e7 and BK on e5).
4. En-passant capture gives discovered check

Steve

[sje]

Those are good suggestions. Please post any corresponding FEN strings if you have them.

[Steve Maughan]

Hi Steven,

Try these...

En Passant Test:
[d]8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - -

EP, Promotion & Castling Test:
[d]8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K -

Best regards,

Steve

[sje]

Bozo says:

Code: Select all

[] sfen 8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - - 0 1
[] perftbulk 1
PerftBulk depth: 1
FEN: 8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - - 0 1
  Kg1 1
  Kg2 1
  Kh1 1
  Kh3 1
  Rd1 1
  Rd3 1
  Rxd4+ 1
  b3 1
  c3 1
  c4 1
  e3 1
  e4 1
Pathcount: 12
[] perftbulk 2
PerftBulk depth: 2
FEN: 8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - - 0 1
  Kg1 17
  Kg2 17
  Kh1 17
  Kh3 17
  Rd1 17
  Rd3 16
  Rxd4+ 8
  b3 16
  c3 18
  c4 18
  e3 18
  e4 18
Pathcount: 197
[] perftbulk 3
PerftBulk depth: 3
FEN: 8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - - 0 1
  Kg1 221
  Kg2 248
  Kh1 187
  Kh3 203
  Rd1 308
  Rd3 287
  Rxd4+ 146
  b3 169
  c3 237
  c4 207
  e3 262
  e4 221
Pathcount: 2696
[] sfen 8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K - 0 1
[] perftbulk 1
PerftBulk depth: 1
FEN: 8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K - 0 1
  Kd1 1
  Ke2 1
  Kf1 1
  Kf2 1
  O-O+ 1
  Rf1+ 1
  Rg1 1
  d3 1
  d4 1
  f8=B 1
  f8=N 1
  f8=Q+ 1
  f8=R+ 1
  g3+ 1
  g4 1
  h3 1
  h4 1
Pathcount: 17
[] perftbulk 2
PerftBulk depth: 2
FEN: 8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K - 0 1
  Kd1 13
  Ke2 13
  Kf1 13
  Kf2 13
  O-O+ 3
  Rf1+ 3
  Rg1 13
  d3 15
  d4 14
  f8=B 15
  f8=N 15
  f8=Q+ 4
  f8=R+ 4
  g3+ 5
  g4 13
  h3 13
  h4 13
Pathcount: 182
[] perftbulk 3
PerftBulk depth: 3
FEN: 8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K - 0 1
  Kd1 219
  Ke2 277
  Kf1 194
  Kf2 263
  O-O+ 65
  Rf1+ 59
  Rg1 207
  d3 249
  d4 220
  f8=B 278
  f8=N 254
  f8=Q+ 117
  f8=R+ 101
  g3+ 77
  g4 204
  h3 218
  h4 230
Pathcount: 3232

[Ajedrecista]

Hello:

Those positions are so good for perft purposes! Congratulations. I have run JetChess for giving some results... I hope they are good, but a verification by BozoChess and/or other perft counters is welcome, as usual.

[d]8/3k4/8/4b3/1p1p1p2/8/1PPRP2K/8 w - -

Code: Select all

Perft(1) = 12
Perft(2) = 197
Perft(3) = 2,696
Perft(4) = 41,622
Perft(5) = 626,367
Perft(6) = 9,544,506
Perft(7) = 151,575,591
Perft(8) = 2,298,846,030
Perft(9) = 37,578,503,653
Perft(10) = 569,150,752,638

[d]8/4rP2/8/8/4pk2/8/3P2PP/4K2R w K -

Code: Select all

Perft(1) = 17
Perft(2) = 182
Perft(3) = 3,232
Perft(4) = 42,552
Perft(5) = 802,123
Perft(6) = 11,608,216
Perft(7) = 235,264,402
Perft(8) = 3,540,539,758
Perft(9) = 77,015,889,530
Perft(10) = 1,175,668,356,863

Good luck with BozoChess and the node count of Perft(13).

Regards from Spain.

Ajedrecista.

[Uri Blass]

My soon-to-be-released BozoChess program has a built-in self test routine which includes a set of perft test positions. Each position is sent to the perft bulk counter for depths of one to three ply and the results are checked against known values. Since the bulk counter can do move than ten million positions per second on my iMac, there is no noticeable time take during program initialization to run the self test.

But I have only a few positions and would like to add more, and so I'm looking for suggestions. The list so far:

Code: Select all

    fen := 'rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1';
    fen := '2qrr1n1/3b1kp1/2pBpn1p/1p2PP2/p2P4/1BP5/P3Q1PP/4RRK1 w - - 0 1';
    fen := 'r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1';
    fen := '8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 1';
    fen := 'r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1';
    fen := 'r2q1rk1/pP1p2pp/Q4n2/bbp1p3/Np6/1B3NBn/pPPP1PPP/R3K2R b KQ - 0 1';
    fen := 'rnbqkb1r/ppppp1pp/7n/4Pp2/8/8/PPPP1PPP/RNBQKBNR w KQkq f6 0 3';

If the target is to help to have a debugging tool for chess programs then I suggest to have million random positions from real chess games.

If the sum of perft(1) and the sum of perft(2) and the sum of perft(3) on all the positions is correct against known values based on more than one program then it means that the program probably has no bug in the move generator.

[sje]

Thank you for your perft results for Steve's two positions.

But I still need more positions, particularly where a bug is likely to show up in the first three perft ply.

In all testing to date, I have seen only two perft errors in Bozo. The first was due to a typo (I make lots of these) in the castling descriptor table for Black king side castling; this was found quickly. The second was a coding error on my part which allowed a pawn pinned against a king by a queen or rook from behind to be unable to advance; this took a half hour to find and fix.

There will probably be more errors when I get to the checks-only move generator.

[sje]

If the target is to help to have a debugging tool for chess programs then I suggest to have million random positions from real chess games.

The objective here is for Bozo to give a quick, traceable failure if there are obvious bugs in the generate/execute/retract machinery or its initialization code. While I believe that the program is correct, it is likely that some who experiment with the source (as I intend) could introduce changes in the code and that's where the self test will help.

In any case, the assertions in the source code have already been exercised several billion times without failure. This includes a passive king in check test on every execution, a restore fidelity check on every retraction, and a full board legality check prior to every generation. The compiler generated range checking helps, too. Those millions of random games weren't produced just for amusement.

[ibid]

Here is the one that derailed my first attempt to verify perft 11 many years ago
[d]rnb1kbnr/pp1ppppp/8/8/q1p4K/5P2/PPPPP1PP/RNBQ1BNR w kq - 2 5
My engine managed to interpret the revealed check after d4 cxd3+ as a double check and thus botched the following ply.

You might also try something like this, which features an actual double revealed check:
[d]rnbq1bnr/pppp1ppp/3kp3/3P4/1B6/8/PPP1PPPP/RN1QKBNR b KQ - 2 4
Clearly these will only catch engines that try to get too smart with en passant. :)

-paul