Move generator advice

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
mvanthoor
Posts: 1784
Joined: Wed Jul 03, 2019 4:42 pm
Location: Netherlands
Full name: Marcel Vanthoor

Re: Move generator advice

Post by mvanthoor »

RedBedHed wrote: Thu Aug 12, 2021 11:21 pm For some reason it is even faster on KiwiPete and Position 6, yielding upwards of 400,000,000 leaf nps...
I assume you are using bulk counting to reach those speeds. If you are, then be prepared for a _massive_ drop in speed when you are going to implement the chess engine, because:

- Bulk counting is a perft trick, and useless for playing chess
- Your engine is going to have things to keep track of incrementally, such as zobrist hashing, while making moves. This reduces perft speed. After you implement PST's, you'll find that applying the PST's over and over again in the evaulation function is very expensive... so you'll start to keep the PST value incrementally as well. And a second set too, when you add tapered evaluation. And the evaluation phase / material score...

I did the same as you, focusing on the speed of the move generator (without bulk counting, because that isn't used for chess), and each time I added a new feature with incrementally kept values the leaves/second dropped a few percent. And it still hurts :?

One optimization I can still make is looking into PEXT, but I don't know how much that will gain.
Author of Rustic, an engine written in Rust.
Releases | Code | Docs | Progress | CCRL
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: Move generator advice

Post by dangi12012 »

RedBedHed wrote: Thu Aug 12, 2021 11:16 pm Update on speed (now using pext bitboards, if available) :

perft(1) - 0.000 seconds - 20 nodes visited.
perft(2) - 0.000 seconds - 400 nodes visited.
perft(3) - 0.000 seconds - 8902 nodes visited.
perft(4) - 0.000 seconds - 197281 nodes visited.
perft(5) - 0.016 seconds - 4865609 nodes visited.
perft(6) - 0.344 seconds - 119060324 nodes visited.
perft(7) - 9.047 seconds - 3195901860 nodes visited.
perft(8) - 249.016 seconds - 84998978956 nodes visited.

(about 340,000,000 leaf nps at perft 8)
Gigantua.exe "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" 8
Perft 7: 3195901860 2513ms 1271.36 MNodes/s
Perft 8: 84998978956 66791ms 1272.6 MNodes/s

Gigantua.exe "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq -" 6
Perft 5: 193690690 92ms 2105.24 MNodes/s
Perft 6: 8031647685 4291ms 1871.64 MNodes/s


No hashing - Single Thread - Bulk counting enabled. What do you get? https://github.com/Gigantua/Gigantua/releases/tag/v1.40,

If qperft was fastest until 2020 - what is gigantua in 2021? The fastest chess movegen ever.
I have still one idea more that will improve the speed by 10%. Then its as fast as I can go on my own - and then I will make my sourcecode public.

cc0 is running slowly in my wsl. Also its calculating for 9 seconds and then printing: perft(6) - 0.219 seconds??
but looking at sourcecode i have a few tips:
- Checking is correct with &= checkmask.
- Pins can be done similar with &= pinmask. This also solves the nasty Enpassant Pins automatically and for free
- Template parameter not only for white - but also for Enpassant and Castling status. These moves are free because they are rare. So no single if statement should be needed at runtime for a position without it :)
- Movelist is not needed at all. Not even for sorting or for storing moves. That can be done when you visit a move - but much cheaper.
This will all be clear once I release my movegen sourcecode!
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer
R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: Move generator advice

Post by R. Tomasi »

dangi12012 wrote: Thu Sep 30, 2021 11:31 am - Movelist is not needed at all. Not even for sorting or for storing moves. That can be done when you visit a move - but much cheaper.
This will all be clear once I release my movegen sourcecode!
OP ist trying to build an actual engine, afaik, in which case that advice is pure BS.
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: Move generator advice

Post by dangi12012 »

R. Tomasi wrote: Thu Sep 30, 2021 12:54 pm
dangi12012 wrote: Thu Sep 30, 2021 11:31 am - Movelist is not needed at all. Not even for sorting or for storing moves. That can be done when you visit a move - but much cheaper.
This will all be clear once I release my movegen sourcecode!
OP ist trying to build an actual engine, afaik, in which case that advice is pure BS.
"R. Tomasi" still a troll. You have literally NO idea how to do stuff. - You can do move ordering - but you dont need a list for all moves. Finding the best 8 moves out of 30 possible does not need to sort 30 elements. Like finding the max element of an array does not need to sort the whole array.
You just store 8 moves and if you visit a better move than the min of the 8 you replace the weakest one of the 8.

Also a fast movegen (wo bulk count) on its own is useful for any engine.
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer
R. Tomasi
Posts: 307
Joined: Wed Sep 01, 2021 4:08 pm
Location: Germany
Full name: Roland Tomasi

Re: Move generator advice

Post by R. Tomasi »

dangi12012 wrote: Thu Sep 30, 2021 1:18 pm "R. Tomasi" still a troll.
Pointing out BS isn't trolling (whereas hijacking other threads to advertise your money funneling project might be considered so).
dangi12012 wrote: Thu Sep 30, 2021 1:18 pm You have literally NO idea how to do stuff. - You can do move ordering - but you dont need a list for all moves. Finding the best 8 moves out of 30 possible does not need to sort 30 elements. Like finding the max element of an array does not need to sort the whole array.
You just store 8 moves and if you visit a better move than the min of the 8 you replace the weakest one of the 8.
And that exactly, is totally useless in an actual engine. Or do you prefer to repeat the movegen step for every new iteration in an IID framework?
dangi12012 wrote: Thu Sep 30, 2021 1:18 pm Also a fast movegen (wo bulk count) on its own is useful for any engine.
Yes, it is. It's however not what this thread is about.

I suggest you stop accusing other people of trolling just because they point out the obviously false statements in you posts.
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: Move generator advice

Post by dangi12012 »

R. Tomasi wrote: Thu Sep 30, 2021 1:29 pm ...
Completely wrong as usual. I need a fair comparison with other movegens and this perft(5) - 0.031 seconds - 4865609 nodes visited. On a Core i5 seems interesting to me.

So someone can test both movegens and verify which on is fastest on Intel/Ryzen 5000?
Qperft vs cc0 vs giga
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer
klx
Posts: 179
Joined: Tue Jun 15, 2021 8:11 pm
Full name: Emanuel Torres

Re: Move generator advice

Post by klx »

FYI: I have contacted the real "Daniel Inführ" to let him know that someone has created an account in his name, and is doing his best to destroy his reputation. I suspect it's either a colleague or ex-boyfriend who want to hurt him for some reason.
[Moderation warning] This signature violated the rule against commercial exhortations.
tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: Move generator advice

Post by tcusr »

dangi12012 wrote: Thu Sep 30, 2021 11:31 am
RedBedHed wrote: Thu Aug 12, 2021 11:16 pm Update on speed (now using pext bitboards, if available) :

perft(1) - 0.000 seconds - 20 nodes visited.
perft(2) - 0.000 seconds - 400 nodes visited.
perft(3) - 0.000 seconds - 8902 nodes visited.
perft(4) - 0.000 seconds - 197281 nodes visited.
perft(5) - 0.016 seconds - 4865609 nodes visited.
perft(6) - 0.344 seconds - 119060324 nodes visited.
perft(7) - 9.047 seconds - 3195901860 nodes visited.
perft(8) - 249.016 seconds - 84998978956 nodes visited.

(about 340,000,000 leaf nps at perft 8)
Gigantua.exe "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" 8
Perft 7: 3195901860 2513ms 1271.36 MNodes/s
Perft 8: 84998978956 66791ms 1272.6 MNodes/s

Gigantua.exe "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq -" 6
Perft 5: 193690690 92ms 2105.24 MNodes/s
Perft 6: 8031647685 4291ms 1871.64 MNodes/s


No hashing - Single Thread - Bulk counting enabled. What do you get? https://github.com/Gigantua/Gigantua/releases/tag/v1.40,

If qperft was fastest until 2020 - what is gigantua in 2021? The fastest chess movegen ever.
I have still one idea more that will improve the speed by 10%. Then its as fast as I can go on my own - and then I will make my sourcecode public.

cc0 is running slowly in my wsl. Also its calculating for 9 seconds and then printing: perft(6) - 0.219 seconds??
but looking at sourcecode i have a few tips:
- Checking is correct with &= checkmask.
- Pins can be done similar with &= pinmask. This also solves the nasty Enpassant Pins automatically and for free
- Template parameter not only for white - but also for Enpassant and Castling status. These moves are free because they are rare. So no single if statement should be needed at runtime for a position without it :)
- Movelist is not needed at all. Not even for sorting or for storing moves. That can be done when you visit a move - but much cheaper.
This will all be clear once I release my movegen sourcecode!
how can castling rights be used as a template parameter? they are not constexpr, just like en passant squares
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: Move generator advice

Post by dangi12012 »

tcusr wrote: Thu Sep 30, 2021 9:59 pm how can castling rights be used as a template parameter? they are not constexpr, just like en passant squares
You will see tomorrow in the sourcecode if all goes according to plan. Reddit post - Codeproject Article - Github release https://github.com/Gigantua/Gigantua

The juicy bits like hashtable are not implemented yet. I also will do Multithreading just because I want to see big numbers. - but after the release on github. I am also hoping that people here will have good comments that can increase the speed yet again.

Regarding you question: Its possible you can read it tomorrow - I wont spoil the suprise!

Just now I did the first CLANG compile under WSL!
Perft Start 6: 119060324 88ms 1348.04 MNodes/s
Perft Start 7: 3195901860 2171ms 1471.86 MNodes/s

Perft Kiwi 5: 193690690 87ms 2214.27 MNodes/s
Perft Kiwi 6: 8031647685 3906ms 2056.08 MNodes/s
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer