Stockfish "use sleeping threads" - Redux

[adamh]

Hello folks,
I am a total newbie here, so I thought I would start off by contributing a bit...

The discussion in the thread: StockFish "use sleeping threads", got very deep (forgive my pun), and also a bit muddled and IMHO it actually needs three threads
1 the effect of the "use sleeping threads" option ?
2 HyperThreading pros and cons ?
3 is "use sleeping threads" really a well chosen name ?

So I decided to start two new threads for 1 & 2. if I am breaking Chesstalk etiquette let me know, but please go easy on the NKOTB




The tests were run on some serious bit of silicon:
CPU: 2 x 3.33 GHz, 12 physical cores
RAM: Enough
Book: Ryb3
Tourney engine: Fritz, all book exits with "reverse colours"
Ponder: off

Time: 300 x 1 min
StockFish 3.0.1, 16 threads, with "use sleeping threads" 43 wins ( 27w + 16b ) => pts 52.3 % (crashed once)
StockFish 3.0.1, 16 threads, without "use sleeping threads" 29 wins ( 18w + 11b ) => pts 47.7 %
Draws: 228




Time: 300 x 3 min
StockFish 3.0.1, 12 threads, with "use sleeping threads" 35 wins ( 19w + 16b ) => 52.7 pts %
StockFish 3.0.1, 12 threads, without "use sleeping threads" 19 wins ( 14w + 5b ) => 47.3 pts %
Draws: 246


Perhaps it was a bit unscientific to change the number of threads as well as the time control but my hunch is that the effect of 12 vs 16 threads on a 12 core box is marginal...

So the new flag seems indeed to produce an increase in strength, but as somewhere mentioned (Ippolit?), it unfortunately makes SF a bit more crash prone. (During deeeep analysis as well.)

I daresay the the difference between the 3min and 1min games is negligible. But, I remember someone (Marco Costalba?) asking for extensive tests with 10 sec games. I may be a total noob, but I think that there might be a possibility that such short times could produce skewed results (in relation to practical playing strength).

[zullil]

The tests were run on some serious bit of silicon:
CPU: 2 x 3.33 GHz, 12 physical cores
RAM: Enough
Book: Ryb3
Tourney engine: Fritz, all book exits with "reverse colours"
Ponder: off


Time: 300 x 3 min
StockFish 3.0.1, 12 threads, with "use sleeping threads" 35 wins ( 19w + 16b ) => 52.7 pts %
StockFish 3.0.1, 12 threads, without "use sleeping threads" 19 wins ( 14w + 5b ) => 47.3 pts %
Draws: 246


Perhaps it was a bit unscientific to change the number of threads as well as the time control but my hunch is that the effect of 12 vs 16 threads on a 12 core box is marginal...

In your second test, presumably no hyperthreading is occurring. In that case, it's not clear to me why "Use Sleeping Threads" would make any difference at all. Maybe Tord or Marco can comment on this.

[bob]

Hello folks,
I am a total newbie here, so I thought I would start off by contributing a bit...

The discussion in the thread: StockFish "use sleeping threads", got very deep (forgive my pun), and also a bit muddled and IMHO it actually needs three threads
1 the effect of the "use sleeping threads" option ?
2 HyperThreading pros and cons ?
3 is "use sleeping threads" really a well chosen name ?

So I decided to start two new threads for 1 & 2. if I am breaking Chesstalk etiquette let me know, but please go easy on the NKOTB




The tests were run on some serious bit of silicon:
CPU: 2 x 3.33 GHz, 12 physical cores
RAM: Enough
Book: Ryb3
Tourney engine: Fritz, all book exits with "reverse colours"
Ponder: off

Time: 300 x 1 min
StockFish 3.0.1, 16 threads, with "use sleeping threads" 43 wins ( 27w + 16b ) => pts 52.3 % (crashed once)
StockFish 3.0.1, 16 threads, without "use sleeping threads" 29 wins ( 18w + 11b ) => pts 47.7 %
Draws: 228




Time: 300 x 3 min
StockFish 3.0.1, 12 threads, with "use sleeping threads" 35 wins ( 19w + 16b ) => 52.7 pts %
StockFish 3.0.1, 12 threads, without "use sleeping threads" 19 wins ( 14w + 5b ) => 47.3 pts %
Draws: 246


Perhaps it was a bit unscientific to change the number of threads as well as the time control but my hunch is that the effect of 12 vs 16 threads on a 12 core box is marginal...

So the new flag seems indeed to produce an increase in strength, but as somewhere mentioned (Ippolit?), it unfortunately makes SF a bit more crash prone. (During deeeep analysis as well.)

I daresay the the difference between the 3min and 1min games is negligible. But, I remember someone (Marco Costalba?) asking for extensive tests with 10 sec games. I may be a total noob, but I think that there might be a possibility that such short times could produce skewed results (in relation to practical playing strength).

I strongly suspect this test is meaningless. The idea "use sleeping threads" is not something that "parses" in my terminology.

You might try HT on and off without changing this. It _might_ well be that simply setting "use sleeping threads" is a more efficient way to search. I looked at SF 2.0 a bit, but I simply do not do thread stuff the way they are doing things. I have no restriction about a thread helping only below the point where it is a master. In Crafty, any thread can help any other thread, no matter where things are with respect to split points.

It may be that the "use sleeping threads" option therefore simply enables a somewhat more efficient threading approach.

If you ignore that, and try to use N threads with HT off, and then 2N threads with HT on (N = number of physical cores) you will likely measure what you are interested in, namely does HT help or hurt. Dinking around with something else (use sleeping threads on or off) in addition to enabling HT or not is changing two things at once and you can't determine which is responsible for any gain or loss...

[mcostalba]

If you ignore that, and try to use N threads with HT off, and then 2N threads with HT on (N = number of physical cores) you will likely measure what you are interested in, namely does HT help or hurt. Dinking around with something else (use sleeping threads on or off) in addition to enabling HT or not is changing two things at once and you can't determine which is responsible for any gain or loss...

If HT is on then "use sleeping threads" must be on. I hope this is not so difficult to grasp.

Regrading the test I am positively surprised that "use sleeping threads" on works better than "use sleeping threads" off even when "threads" is set equal to the number of physical cores.

Thanks a lot for yout test Adam, it is very interesting and important result for me. Yes 10" TC is probably too short, regarding this is interesting that results seem do not change with TC, another thing about which people was sceptical.

[bob]

If you ignore that, and try to use N threads with HT off, and then 2N threads with HT on (N = number of physical cores) you will likely measure what you are interested in, namely does HT help or hurt. Dinking around with something else (use sleeping threads on or off) in addition to enabling HT or not is changing two things at once and you can't determine which is responsible for any gain or loss...

If HT is on then "use sleeping threads" must be on. I hope this is not so difficult to grasp.

Very difficult to grasp for me since I see no reason to differentiate between HT on and off inside the chess engine. Concept makes no sense to me at all. If it is a spinlock issue, there is a better solution based on the "pause" instruction. Otherwise I don't see anything to make HT on vs HT off get different treatment in a chess engine. It certainly is not in mine...

BTW, is this a bit of perhaps broken English? Does it mean "threads sleep when waiting rather than spinning"? for example?

Regrading the test I am positively surprised that "use sleeping threads" on works better than "use sleeping threads" off even when "threads" is set equal to the number of physical cores.

Thanks a lot for yout test Adam, it is very interesting and important result for me.

[mcostalba]

I strongly suspect this test is meaningless. The idea "use sleeping threads" is not something that "parses" in my terminology.

Sorry but I fail to see a link between the two sentences.

P.S: Regrading terminology, in case you may want to give a not so quick look at the code, perhaps then is gonna parse better

[mcostalba]

BTW, is this a bit of perhaps broken English? Does it mean "threads sleep when waiting rather than spinning"? for example?

yes, that's the meaning. I am not native speaker so probably I choose a not best name, but for me a thread blocked on a condition variable "does sleep".

BTW I have given a quick look at Crafty code but the "pause" solution does not seem portable to me becuase is coded in an asm{} gcc-ism statement that does not work under Windows....

[Houdini]

BTW I have given a quick look at Crafty code but the "pause" solution does not seem portable to me becuase is coded in an asm{} gcc-ism statement that does not work under Windows....

In Windows there's the function YieldProcessor() - maybe this only works in Visual Studio.

Robert

[mcostalba]

BTW I have given a quick look at Crafty code but the "pause" solution does not seem portable to me becuase is coded in an asm{} gcc-ism statement that does not work under Windows....

In Windows there's the function YieldProcessor() - maybe this only works in Visual Studio.

Robert

From
http://msdn.microsoft.com/en-us/library ... 85%29.aspx

Code: Select all

YieldProcessor Macro

Signals to the processor to give resources to threads that are waiting for them. This macro is only effective on processors that support technology allowing multiple threads running on a single processor, such as Intel's Hyperthreading technology.

I am not sure is an equivalent solution, becuase it seems if HT is off then threads keep spinning....at least reading the docs it seems like this to me.

[adamh]

I posted a reply to Robert but that vanished somehow, probably I goofed. Anyway, most of what I wrote was rendered irrelevant by the subsequent exchange.

So Marco: this option it uses thread synchronization mechanisms to start and stop threads? Contrary to what? Terminating and creating new threads? Or using variable polling? Have I understood anything?

And maybe something like "Use thread-waiting" or "Use thread synchronization objects" will make me (and others) shut up about terminology?