What is a good perft speed?

Discussion of chess software programming and technical issues.

Moderators: hgm, chrisw, Rebel

User avatar
Ras
Posts: 2638
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: What is a good perft speed?

Post by Ras »

lithander wrote: Thu Dec 21, 2023 1:51 pmI tried tricks like that but couldn't get any significant nps increase out of it (for search). The one you describe only works when not already in check.
Correct, my approach doesn't work for in-check positions (but I have evasions for that), king moves, and I also exclude en passant moves. Still, I got a measurable speedup, IIRC something like 10% or so. The reason is that my move generator and in-check test work with for-loops (sliders) and dedicated if-checks (knights). If your engine has a more efficient implementation to begin with, the spared work and hence the speed-up will be smaller.
Rasmus Althoff
https://www.ct800.net
User avatar
Ajedrecista
Posts: 2019
Joined: Wed Jul 13, 2011 9:04 pm
Location: Madrid, Spain.

Re: What is a good perft speed?

Post by Ajedrecista »

Hello:
Ajedrecista wrote: Wed Dec 20, 2023 7:34 pm[...]

So I ran benchmarks of Perft(7) of the starting position in my PC with SF 16, qperft, JetChess 1.0.0.0 and gperft 1.1 to compare:

Code: Select all

[...]

=======================================================================
ENGINE / COUNTER        HASH (MB)        THREADS        TIME (M:SS.sss)
=======================================================================
gperft 1.1                 64               2           0:00.815
gperft 1.1                 64               2           0:00.866
[...]
[...]
There is a typo at the top of the chart that I want to correct:

Code: Select all

[...]

=======================================================================
ENGINE / COUNTER        HASH (MB)        THREADS        TIME (M:SS.sss)
=======================================================================
gperft 1.1                 64               2           0:00.815
gperft 1.1                128               2           0:00.866
[...]
I wrongly duplicated 64 MB of hash when writting.

Regards from Spain.

Ajedrecista.
eduherminio
Posts: 75
Joined: Mon Apr 05, 2021 12:00 am
Full name: Eduardo Caceres

Re: What is a good perft speed?

Post by eduherminio »

kranium wrote: Thu Dec 21, 2023 12:46 am I've updated the list
https://github.com/FireFather/perft-speeds

Now it's an editable table instead of a bitmap.

I'll be happy to add engines...
just contact me here firefather at telenet dot be
and send a copy of or point me to your engine.
Isn't the used fen/position quite a relevant piece of information to include?
Author of Lynx chess engine (GitHub, Lichess)
chessbit
Posts: 6
Joined: Fri Dec 29, 2023 4:47 pm
Location: Belgium
Full name: thomas albert

Re: What is a good perft speed?

Post by chessbit »

imnoumni wrote: Tue Dec 19, 2023 11:20 pm

Code: Select all

$ time ./main divide startpos 7                                                                                                                        
... moves ...
perft 3195901860 nps 639087301 time 5.000728

real    0m5.003s
user    0m5.003s
sys     0m0.000s
This is pretty good. Could you tell us what approach you used? I assume bitboards, perhaps with pext?
Did you move some logic to compile time?
My engine is about 50% slower with only run time but I don't find many ways to improve :(
User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: What is a good perft speed?

Post by mvanthoor »

639 million NPS... that's a lot. I assume you use bulk counting and/or a transposition table, and maybe other perft-specific tricks.

My engine runs at +/- 58 million NPS, on a Ryzen 7950X, without a hash table, multi-threading or bulk counting. On an older i7 6700K, it runs at 32 million NPS. It does full incremental updates for its evaluation and Zobrist key when running perft; I do not disable this (because I mainly use perft to see if everything is working as it should.)

When running with a hash-table, speed increases the deeper it goes (obviously, because more and more depths can be pulled directly from the TT):

With 256 MB TT:

Code: Select all

Perft 1: 20 (0 ms, inf leaves/sec, hash full: 0%)
Perft 2: 400 (0 ms, inf leaves/sec, hash full: 0%)
Perft 3: 8902 (0 ms, inf leaves/sec, hash full: 0%)
Perft 4: 197281 (3 ms, 65760333 leaves/sec, hash full: 0%)
Perft 5: 4865609 (58 ms, 83889810 leaves/sec, hash full: 1%)
Perft 6: 119060324 (819 ms, 145372800 leaves/sec, hash full: 13.3%)
Perft 7: 3195901860 (10874 ms, 293903058 leaves/sec, hash full: 90.3%)
Total time spent: 11754 ms
Execution speed: 282459962 leaves/second
Average at 282 million NPS/second, but because this will keep increasing, I doubt it actually says anything. Someday I'd have to make this multi-threaded to see how fast it will go. I did an experiment a few years ago when just developing the engine, and it worked, but it wasn't the best way of multi-threading perft.
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
rdhoffmann
Posts: 36
Joined: Fri Apr 21, 2023 3:46 pm
Full name: Richard Hoffmann

Re: What is a good perft speed?

Post by rdhoffmann »

Just 5 seconds is amazingly fast, ihmo.

I tried the move generator from CPW (0x88 board representation) and it takes about 10 sec here just to generate the pseudo-legal moves alone for perft(7). So a full perft implementation based on that code should be much slower than Stockfish and of course your code. I don't know if Stockfish uses hash tables when doing perft though.
User avatar
phhnguyen
Posts: 1501
Joined: Wed Apr 21, 2010 4:58 am
Location: Australia
Full name: Nguyen Hong Pham

Re: What is a good perft speed?

Post by phhnguyen »

rdhoffmann wrote: Fri Nov 22, 2024 9:47 am I tried the move generator from CPW (0x88 board representation) and it takes about 10 sec here just to generate the pseudo-legal moves alone for perft(7). So a full perft implementation based on that code should be much slower than Stockfish and of course your code. I don't know if Stockfish uses hash tables when doing perft though.
Once I have studied this "issue".

Both my program (Felicity EGTB) and CPW-engine use the mailbox for the board representation. Similar to CPW-engine, my Perft speed was significantly slower than Stockfish. In my first measure, the Perft speed was 31 times slower. After some cleaning and optimising, it was about 14 times slower.

If we want to compare pure Perft speeds, we can stop here since the results are so clear. However, via Perft speeds and besides checking the correctness, almost all of us mean to compare the speeds of move generators, and functions of making/taking back/in-check... We need to look deeper. For me, the result is a bit shocking and not reasonable because my code is very simple, clean, and straightforward thus it should be very fast too, especially in simple tasks such as Perft.

It turns out the way we measure Perft is different. SF with pre-computed data (thus it can generate all legal moves very fast without making/taking back moves) can use the method of Bulk-counting to avoid visiting (save making/taking back moves) for the last level. In contrast, normal mailbox programs must make/take back moves for all moves, including the last level, to make sure all nodes are legal. That is why mine is much slower. In other words, we have compared the orange and apple.

To be fair, we should use the same way. I have fixed quickly the SF to use the same method as mine and SF is only 1.7 times faster. I think the last result is reasonable, SF uses some pre-calculated data and has more optimised functions thus it should be a bit faster. I can make mine a bit faster too if I use some more tricks and pre-computed data too if I want.

You may read my document about Perft attempts here!
https://banksiagui.com
The most features chess GUI, based on opensource Banksia - the chess tournament manager
User avatar
hgm
Posts: 28216
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: What is a good perft speed?

Post by hgm »

Qperft is mailbox, but it does not have to make/unmake all the moves for determining whether they are legal. Only King moves. The rest are bulk-counted, after 'bulk legality testing'.
rdhoffmann
Posts: 36
Joined: Fri Apr 21, 2023 3:46 pm
Full name: Richard Hoffmann

Re: What is a good perft speed?

Post by rdhoffmann »

Thanks phhnguyen, that explains it. As a quick test with make/unmake and a legality check, I get about 50 seconds now for perft(7) using the mailbox board rep and CPW move generator. This is still 10x slower than what OP wrote though.
Dann Corbit
Posts: 12699
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: What is a good perft speed?

Post by Dann Corbit »

Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.