Page 2 of 2

Re: Simplest way to implement quick and dirty lazy smp

Posted: Sun Jan 06, 2019 1:30 pm
by syzygy
lucasart wrote: Sun Jan 06, 2019 1:44 am I fail to see how this is easier.
Easier than rewriting the whole engine to make it thread safe...

Re: Simplest way to implement quick and dirty lazy smp

Posted: Sun Jan 06, 2019 2:41 pm
by xr_a_y
syzygy wrote: Sun Jan 06, 2019 1:30 pm
lucasart wrote: Sun Jan 06, 2019 1:44 am I fail to see how this is easier.
Easier than rewriting the whole engine to make it thread safe...
Appart from looking for static (or global) variables and making some table member of the "thread context", there is not much to make it thread safe I guess.

Re: Simplest way to implement quick and dirty lazy smp

Posted: Sun Jan 06, 2019 8:28 pm
by hgm
JVMerlino wrote: Sat Jan 05, 2019 10:23 pmThe only significant difference between your implementation and mine is that I pass the address for the hash tables as commandline parameters to the slaves. How do you pass commands down the chain if not via pipes?
Oh, I understood that you had one master process that would have all other processes as child, and communicate directly to them. Sorry if I was wrong. And yes, I do pass along the commands through pipes. I just used the same code as that WinBoard and adapters use to fork off an engine process, and set up the pipes to it.
lucasart wrote: Sun Jan 06, 2019 1:44 amI fail to see how this is easier. Honestly, managing subprocesses, and communications via pipes, is a lot more complicated in my experience (especially on POSIX systems, with the fork logic being quite mind bending). And it's hideous, from a design/coding standpoint. Simply start threads in an id_loop() and join when done. That's all there is to it. The rest is details, which you can improve on as you go along.
It would not be easier for an engine that you design from scratch, of course. But if you already have a single-threaded engine, using many global variables, it would require a lot of modifications to even run two searches in parallel. As it was I only required to copy-paste existing code for starting an engine process, and some minor modifications to the command-interpreter loop (re-ordering the command matching so that all those that do not have to be passed along are matched first, and the rest only after the input has been passed along, change the code that handles the hash-table allocation to use shared memory, and have the execution of a few specific commands pass something special). Much less error prone. I did the entire conversion in the plane while flying towards the tournament.

Re: Simplest way to implement quick and dirty lazy smp

Posted: Mon Jan 07, 2019 1:20 pm
by lucasart
hgm wrote: Sun Jan 06, 2019 8:28 pm
lucasart wrote: Sun Jan 06, 2019 1:44 amI fail to see how this is easier. Honestly, managing subprocesses, and communications via pipes, is a lot more complicated in my experience (especially on POSIX systems, with the fork logic being quite mind bending). And it's hideous, from a design/coding standpoint. Simply start threads in an id_loop() and join when done. That's all there is to it. The rest is details, which you can improve on as you go along.
It would not be easier for an engine that you design from scratch, of course. But if you already have a single-threaded engine, using many global variables, it would require a lot of modifications to even run two searches in parallel. As it was I only required to copy-paste existing code for starting an engine process, and some minor modifications to the command-interpreter loop (re-ordering the command matching so that all those that do not have to be passed along are matched first, and the rest only after the input has been passed along, change the code that handles the hash-table allocation to use shared memory, and have the execution of a few specific commands pass something special). Much less error prone. I did the entire conversion in the plane while flying towards the tournament.
Putting lipstick on a pig is never a good long term solution in chess engine development :lol:

If global variables are the problem, and you are looking for a cheap and easy solution, then just put "thread_local" in front of your globals, and voila. Forget about performance on Windows though (Linux implements a zero cost model for TLS, but Windows implementation is crap as always).

In fact, that's how I started with Demolito. Eventually I made it available for windows, and that's how I discovered the cost of TLS on Windows makes it unusable in a chess engine.

Eventually you refactor the code to get rid of most of the globals (ie. "thread_local"), and finally you put the remaining ones in per thread struct, and you have a clean code base.

A good starting point is to question all these global variables you have. How many of them are both read and written during search? Probably very few.

Re: Simplest way to implement quick and dirty lazy smp

Posted: Mon Jan 07, 2019 1:41 pm
by hgm
Long term solutions are of little use when the available time is short...