strategies for finding slowdows in lazy smp

[smatovic]

Currently my main thread is the master-thread. If that one decides the search is finished, then all others terminate as well.

And what happens if a helper finishes its search?

It goes on with the next iteration if applicable. Else it'll busy-loop until the main-thread catches up.

Hmm, not sure, but such an busy-loop could cause your nps drop.

--
Srdja

[flok]

Currently my main thread is the master-thread. If that one decides the search is finished, then all others terminate as well.

And what happens if a helper finishes its search?

It goes on with the next iteration if applicable. Else it'll busy-loop until the main-thread catches up.

Hmm, not sure, but such an busy-loop could cause your nps drop.

Could be but this won't happen at the start-position, only when detecting mate/draw.

[Dann Corbit]

I guess that if your threads quit working when they complete the depth, and the parent monitor waits for all of them to complete, you would have loss for that reason. It would also explain the greater loss with more threads.

Here is an idea:
When a thread finishes the required depth iteration, if it has not received a stop signal, have the thread do a winfinder zero window search 1/2 pawn above the current best score, and just keep going deeper and deeper.

It could still signal the main thread that it is done before it starts the winfinder search, and it would be doing something useful.

If my theory is correct, then longer time control searches will show the NPS loss less and less with longer and longer time control with your current code base.

Your code won't compile on Windows even under msys2 + gcc, so I can only experiment on my UNIX boxes which are not at my disposal right now.

[Sesse]

A profiler might also be helpful. You could try OProfile for Linux (http://oprofile.sourceforge.net/news/).

oprofile? Is this 1999?

perf is really what you want these days; it has basically all the mindshare. If you want something more GUI-ish and commercial, there's stuff like VTune.

[syzygy]

Now my question is: what are strategies for finding what causes this slow down?
The threads share no common variables apart from the transposition table. That tt has no locks, it uses the xor-trick.

Most likely the threads share common cache lines, which has the same effect as sharing common variables.

[flok]

Now my question is: what are strategies for finding what causes this slow down?
The threads share no common variables apart from the transposition table. That tt has no locks, it uses the xor-trick.

Most likely the threads share common cache lines, which has the same effect as sharing common variables.

But why doesn't stockfish have this problem?

[syzygy]

Now my question is: what are strategies for finding what causes this slow down?
The threads share no common variables apart from the transposition table. That tt has no locks, it uses the xor-trick.

Most likely the threads share common cache lines, which has the same effect as sharing common variables.

But why doesn't stockfish have this problem?

The Stockfish threads do not share common cache lines. Apart from the TT, but shared TT cache lines are rare enough. The problem occurs when two or more threads keep reading and writing the same cache line.

However, it seems that in your case the slowdown occurs only when there are (almost) as many search threads as processor threads. Maybe it is not false or true sharing of cache lines but background tasks that are consuming cpu.