Perft function too slow because of vcruntime140.dll

[mvanthoor]

I have this policy for FabChess aswell, but it terribly failed in multithreaded search. Upto a certain amount of threads, let's say 4-8, one is fine with safe Rust, but for TCEC with 176 Threads I had to wrap the cache and certain shared fields with unsafe. Again though, the idea here is to provide a safe api over the unsafe features, spending effort ensuring that the unsafe code is in fact, safe.

Why should that be necessary? If you have a huge amount of threads and you then need to drop back to writing unsafe code (for which reason?) it defeats the purpose of the language. IMHO. It shouldn't be necessary. I hope this improves in the future.

I know that it's possible to provide safe API's over unsafe code. It's basically what C/C++ and other languages have been trying to do with encapsulation. My C/C++ code has always been very safe; my coding style actually fits Rust's paradigm to a T. I never encountered the borrow checker after day 2, after I just learned to put "mut" or "&mut" before a variable I actually wanted to change.

I'm going to think about the unsafe version of the move list. I just realized that even 128 move lists aren't enough. Stockfish, in some endgames, approaches depths of 128; on very fast computers or on huge numbers of cores, it might already exceed it. So, I built a way for the move list pool to expand when its limits are approached (or even exceeded; I'm testing this with a very small pool, where Perft already starts at depth 6-7 and then works backwards), but at that point, there will be an expensive slowdown.

Just doing "let mut list [Move; 256]" is not going to work, because taking this allocation out of perft sped it up from finding 10 million leaf nodes per second to finding 30 million per second.

[Ras]

Why should that be necessary?

Because the hashtable won't work with that many threads and exclusive access. The engine would spend all of its time waiting for the access right to be passed around. Splitting the hashtable into exclusive per-thread chunks won't work either because the hashtable is where the threads exchange search data. You need concurrent, racy access, and preventing that is a core feature of Rust.

[fabianVDW]

Why should that be necessary?

Because the hashtable won't work with that many threads and exclusive access. The engine would spend all of its time waiting for the access right to be passed around. Splitting the hashtable into exclusive per-thread chunks won't work either because the hashtable is where the threads exchange search data. You need concurrent, racy access, and preventing that is a core feature of Rust.

Exactly. And actually, sometime data races aren't a bad thing. Or atleast in this context, they are the least worst thing that can happen to us, as we can easily still validate correctness of data.

[mvanthoor]

Why should that be necessary?

Because the hashtable won't work with that many threads and exclusive access. The engine would spend all of its time waiting for the access right to be passed around. Splitting the hashtable into exclusive per-thread chunks won't work either because the hashtable is where the threads exchange search data. You need concurrent, racy access, and preventing that is a core feature of Rust.

Thanks for pointing that out. So, I assume you implemented lazy SMP with a shared hash table; exactly what I'd do at some point. To be honest, I don't think I'm even going to look into multi-threading until my engine hits 2850 or so.

I have never written anything multi-threaded with that many threads. I'll be sure to look into this when the time arrives. In some cases, indeed, race conditions aren't always bad, especially if data does not have to be perfectly correct at each point in time.

edit: this evening I wrote a function to execute all the perft tests in "perftsuite.epd" as posted here (many other engines and authors include it, by the way):
https://github.com/TerjeKir/Chess-test- ... tsuite.epd

It verified all 127 positions without errors. Many things in there such as pawn edge cases, multiple bishops, etc...

I included a "perftsuite" module in my "extra" module, which isn't compiled into the engine on a normal compile, and basically intend to use it only when I change something with regard to the move generator or move lists in order to try and speed them up.

I consider perft testing to be done for now. The one thing I'd like to include at some point is the hash table, to verify correct calculation of the zobrist keys in make_move.

[fabianVDW]

edit: this evening I wrote a function to execute all the perft tests in "perftsuite.epd" as posted here (many other engines and authors include it, by the way):
https://github.com/TerjeKir/Chess-test- ... tsuite.epd

It verified all 127 positions without errors. Many things in there such as pawn edge cases, multiple bishops, etc...

I included a "perftsuite" module in my "extra" module, which isn't compiled into the engine on a normal compile, and basically intend to use it only when I change something with regard to the move generator or move lists in order to try and speed them up.

I consider perft testing to be done for now. The one thing I'd like to include at some point is the hash table, to verify correct calculation of the zobrist keys in make_move.

Good job on getting things right with perft! Btw. do you have a github link to your project? I couldn't find any on the website nor here on talkchess.

[mvanthoor]

Thanks

Yes, the project is in a private github repo. There's also a website address reserved for documentation.

As soon as the engine can play a few thousand ganes without errors in a GUI, it'll be open sourced as Rustic Alpha 1 on github and its own website so it can be tested for CCRL, CEGT, etc.

I don't expect it to succeed 1400 ELO or so in the lists at that point, because it will have no functionality aside from alpha/beta, move generation and basic time management.

After that, versions will be Alpha 2, 3, etc... until I think the engine is strong enough to become Rustic 1.0. (Maybe 2100 ELO or so.)

I've decided to keep it in safe Rust as long as possible, and then start to use unsafe code if absolutely necessary. It will become a new major version at that point.