Easier than rewriting the whole engine to make it thread safe...
Simplest way to implement quick and dirty lazy smp
Moderators: hgm, Rebel, chrisw
-
- Posts: 5569
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Simplest way to implement quick and dirty lazy smp
-
- Posts: 1871
- Joined: Sat Nov 25, 2017 2:28 pm
- Location: France
Re: Simplest way to implement quick and dirty lazy smp
Appart from looking for static (or global) variables and making some table member of the "thread context", there is not much to make it thread safe I guess.
-
- Posts: 27870
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Simplest way to implement quick and dirty lazy smp
Oh, I understood that you had one master process that would have all other processes as child, and communicate directly to them. Sorry if I was wrong. And yes, I do pass along the commands through pipes. I just used the same code as that WinBoard and adapters use to fork off an engine process, and set up the pipes to it.
It would not be easier for an engine that you design from scratch, of course. But if you already have a single-threaded engine, using many global variables, it would require a lot of modifications to even run two searches in parallel. As it was I only required to copy-paste existing code for starting an engine process, and some minor modifications to the command-interpreter loop (re-ordering the command matching so that all those that do not have to be passed along are matched first, and the rest only after the input has been passed along, change the code that handles the hash-table allocation to use shared memory, and have the execution of a few specific commands pass something special). Much less error prone. I did the entire conversion in the plane while flying towards the tournament.lucasart wrote: ↑Sun Jan 06, 2019 1:44 amI fail to see how this is easier. Honestly, managing subprocesses, and communications via pipes, is a lot more complicated in my experience (especially on POSIX systems, with the fork logic being quite mind bending). And it's hideous, from a design/coding standpoint. Simply start threads in an id_loop() and join when done. That's all there is to it. The rest is details, which you can improve on as you go along.
-
- Posts: 3232
- Joined: Mon May 31, 2010 1:29 pm
- Full name: lucasart
Re: Simplest way to implement quick and dirty lazy smp
Putting lipstick on a pig is never a good long term solution in chess engine developmenthgm wrote: ↑Sun Jan 06, 2019 8:28 pmIt would not be easier for an engine that you design from scratch, of course. But if you already have a single-threaded engine, using many global variables, it would require a lot of modifications to even run two searches in parallel. As it was I only required to copy-paste existing code for starting an engine process, and some minor modifications to the command-interpreter loop (re-ordering the command matching so that all those that do not have to be passed along are matched first, and the rest only after the input has been passed along, change the code that handles the hash-table allocation to use shared memory, and have the execution of a few specific commands pass something special). Much less error prone. I did the entire conversion in the plane while flying towards the tournament.lucasart wrote: ↑Sun Jan 06, 2019 1:44 amI fail to see how this is easier. Honestly, managing subprocesses, and communications via pipes, is a lot more complicated in my experience (especially on POSIX systems, with the fork logic being quite mind bending). And it's hideous, from a design/coding standpoint. Simply start threads in an id_loop() and join when done. That's all there is to it. The rest is details, which you can improve on as you go along.
If global variables are the problem, and you are looking for a cheap and easy solution, then just put "thread_local" in front of your globals, and voila. Forget about performance on Windows though (Linux implements a zero cost model for TLS, but Windows implementation is crap as always).
In fact, that's how I started with Demolito. Eventually I made it available for windows, and that's how I discovered the cost of TLS on Windows makes it unusable in a chess engine.
Eventually you refactor the code to get rid of most of the globals (ie. "thread_local"), and finally you put the remaining ones in per thread struct, and you have a clean code base.
A good starting point is to question all these global variables you have. How many of them are both read and written during search? Probably very few.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
-
- Posts: 27870
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: Simplest way to implement quick and dirty lazy smp
Long term solutions are of little use when the available time is short...