What is a good perft speed?

[Ras]

I tried tricks like that but couldn't get any significant nps increase out of it (for search). The one you describe only works when not already in check.

Correct, my approach doesn't work for in-check positions (but I have evasions for that), king moves, and I also exclude en passant moves. Still, I got a measurable speedup, IIRC something like 10% or so. The reason is that my move generator and in-check test work with for-loops (sliders) and dedicated if-checks (knights). If your engine has a more efficient implementation to begin with, the spared work and hence the speed-up will be smaller.

[Ajedrecista]

Hello:

[...]

So I ran benchmarks of Perft(7) of the starting position in my PC with SF 16, qperft, JetChess 1.0.0.0 and gperft 1.1 to compare:

Code: Select all

[...]

=======================================================================
ENGINE / COUNTER        HASH (MB)        THREADS        TIME (M:SS.sss)
=======================================================================
gperft 1.1                 64               2           0:00.815
gperft 1.1                 64               2           0:00.866
[...]

[...]

There is a typo at the top of the chart that I want to correct:

Code: Select all

[...]

=======================================================================
ENGINE / COUNTER        HASH (MB)        THREADS        TIME (M:SS.sss)
=======================================================================
gperft 1.1                 64               2           0:00.815
gperft 1.1                128               2           0:00.866
[...]

I wrongly duplicated 64 MB of hash when writting.

Regards from Spain.

Ajedrecista.

[eduherminio]

I've updated the list
https://github.com/FireFather/perft-speeds

Now it's an editable table instead of a bitmap.

I'll be happy to add engines...
just contact me here firefather at telenet dot be
and send a copy of or point me to your engine.

Isn't the used fen/position quite a relevant piece of information to include?

[chessbit]

Code: Select all

$ time ./main divide startpos 7                                                                                                                        
... moves ...
perft 3195901860 nps 639087301 time 5.000728

real    0m5.003s
user    0m5.003s
sys     0m0.000s

This is pretty good. Could you tell us what approach you used? I assume bitboards, perhaps with pext?
Did you move some logic to compile time?
My engine is about 50% slower with only run time but I don't find many ways to improve

[mvanthoor]

639 million NPS... that's a lot. I assume you use bulk counting and/or a transposition table, and maybe other perft-specific tricks.

My engine runs at +/- 58 million NPS, on a Ryzen 7950X, without a hash table, multi-threading or bulk counting. On an older i7 6700K, it runs at 32 million NPS. It does full incremental updates for its evaluation and Zobrist key when running perft; I do not disable this (because I mainly use perft to see if everything is working as it should.)

When running with a hash-table, speed increases the deeper it goes (obviously, because more and more depths can be pulled directly from the TT):

With 256 MB TT:

Code: Select all

Perft 1: 20 (0 ms, inf leaves/sec, hash full: 0%)
Perft 2: 400 (0 ms, inf leaves/sec, hash full: 0%)
Perft 3: 8902 (0 ms, inf leaves/sec, hash full: 0%)
Perft 4: 197281 (3 ms, 65760333 leaves/sec, hash full: 0%)
Perft 5: 4865609 (58 ms, 83889810 leaves/sec, hash full: 1%)
Perft 6: 119060324 (819 ms, 145372800 leaves/sec, hash full: 13.3%)
Perft 7: 3195901860 (10874 ms, 293903058 leaves/sec, hash full: 90.3%)
Total time spent: 11754 ms
Execution speed: 282459962 leaves/second

Average at 282 million NPS/second, but because this will keep increasing, I doubt it actually says anything. Someday I'd have to make this multi-threaded to see how fast it will go. I did an experiment a few years ago when just developing the engine, and it worked, but it wasn't the best way of multi-threading perft.

[rdhoffmann]

Just 5 seconds is amazingly fast, ihmo.

I tried the move generator from CPW (0x88 board representation) and it takes about 10 sec here just to generate the pseudo-legal moves alone for perft(7). So a full perft implementation based on that code should be much slower than Stockfish and of course your code. I don't know if Stockfish uses hash tables when doing perft though.

[phhnguyen]

I tried the move generator from CPW (0x88 board representation) and it takes about 10 sec here just to generate the pseudo-legal moves alone for perft(7). So a full perft implementation based on that code should be much slower than Stockfish and of course your code. I don't know if Stockfish uses hash tables when doing perft though.

Once I have studied this "issue".

Both my program (Felicity EGTB) and CPW-engine use the mailbox for the board representation. Similar to CPW-engine, my Perft speed was significantly slower than Stockfish. In my first measure, the Perft speed was 31 times slower. After some cleaning and optimising, it was about 14 times slower.

If we want to compare pure Perft speeds, we can stop here since the results are so clear. However, via Perft speeds and besides checking the correctness, almost all of us mean to compare the speeds of move generators, and functions of making/taking back/in-check... We need to look deeper. For me, the result is a bit shocking and not reasonable because my code is very simple, clean, and straightforward thus it should be very fast too, especially in simple tasks such as Perft.

It turns out the way we measure Perft is different. SF with pre-computed data (thus it can generate all legal moves very fast without making/taking back moves) can use the method of Bulk-counting to avoid visiting (save making/taking back moves) for the last level. In contrast, normal mailbox programs must make/take back moves for all moves, including the last level, to make sure all nodes are legal. That is why mine is much slower. In other words, we have compared the orange and apple.

To be fair, we should use the same way. I have fixed quickly the SF to use the same method as mine and SF is only 1.7 times faster. I think the last result is reasonable, SF uses some pre-calculated data and has more optimised functions thus it should be a bit faster. I can make mine a bit faster too if I use some more tricks and pre-computed data too if I want.

You may read my document about Perft attempts here!

[hgm]

Qperft is mailbox, but it does not have to make/unmake all the moves for determining whether they are legal. Only King moves. The rest are bulk-counted, after 'bulk legality testing'.

[rdhoffmann]

Thanks phhnguyen, that explains it. As a quick test with make/unmake and a legality check, I get about 50 seconds now for perft(7) using the mailbox board rep and CPW move generator. This is still 10x slower than what OP wrote though.

[Dann Corbit]

here is a fast perft
https://github.com/ankan-ban/perft_gpu