How can I switch off hyperthreading?

[Lavir]

The model is Acer Aspire M3970.

The bios on the Acer Aspire M3970 has not an option to disable HT, sadly.

But you don't need it. Run the exe (I presume winboard.exe) with affinities, to use only physical CPUs and you will be fine.

For example, for 4 cores, 8 threads, start a command prompt inside the folder of winboard and type:

Start /affinity 55 winboard.exe

55 is the hexadecimal representation of 01010101 (from core 7 to 0), i.e. core 0, 2, 4, 6

The children the program (winboard) spawns (in this case the exe of the engines) willl automatically inherit the same affinities.

[hgm]

Ah, that is a nice solution. But how can I be sure how the numbers map on physical cores. You seem to use 0,1 -> core 1; 2,3 -> core 2 etc., but when Googling I found other places where it was claimed 0,4 -> core 1; 1,5 -> core 2 etc.

[Houdini]

I am running 4 matches in parallel to test modifications of my engine. I test each version on two matches. What I typically see is that on match A-B gives a very significant advantage for B (say), like 58% over 100 games, while the other A-B is dead even (like 51-49).

I want to rule out the possibility that this is because A in the first match got to share a core with some (also badly losing) A from a match A-C, while B and C happened to be assigned a core of their own.

Windows takes care of that. It is aware of Intel hyper-threading, and will schedule threads preferably one per physical core.
I don't think you should worry about this.

Robert

[JuLieN]

Ah, that is a nice solution. But how can I be sure how the numbers map on physical cores. You seem to use 0,1 -> core 1; 2,3 -> core 2 etc., but when Googling I found other places where it was claimed 0,4 -> core 1; 1,5 -> core 2 etc.

Have you checked my link on the previous page? All the answers were in the comments.

[rbarreira]

I am running 4 matches in parallel to test modifications of my engine. I test each version on two matches. What I typically see is that on match A-B gives a very significant advantage for B (say), like 58% over 100 games, while the other A-B is dead even (like 51-49).

I want to rule out the possibility that this is because A in the first match got to share a core with some (also badly losing) A from a match A-C, while B and C happened to be assigned a core of their own.

Windows takes care of that. It is aware of Intel hyper-threading, and will schedule threads preferably one per physical core.
I don't think you should worry about this.

Robert

Yeah that was also my thought.

I would hope (but I'm not sure) Windows is also smart enough to rotate other threads (such as OS services) to other cores from time to time, in order to be fair to the CPU-intensive threads. But even if this is not the case, the impact should not be that big.

[Lavir]

Ah, that is a nice solution. But how can I be sure how the numbers map on physical cores. You seem to use 0,1 -> core 1; 2,3 -> core 2 etc., but when Googling I found other places where it was claimed 0,4 -> core 1; 1,5 -> core 2 etc.

It depends if you use hex or decimal parameters since you can use both with setaffinity. I prefer to use hex masks because they are more straightforward. (Examples: 4 cores/8 threads: affinity 55; 3 cores/6 threads: affinity 15; 2 cores/4 threads: affinity 5)

Just open calculator and simply write in binary the mask representing all threads and then convert it in hex, very simple.

Anyway, as Robert and others pointed out, if you have an OS as W7, for example, management of threads is done automatically very well (even more so on W8) so you usually don't need to do all of this at all.

However, if you need to be sure, that's a way.

[Eelco de Groot]

Would it not be preferable to run only A-B matches at anyone time, if necessary four matches A-B at a time, then four B-C etc.? There is still a chance of mismatched cores, but you could let loose statistics on the four results to see what is the chance that the four matches were not independant. At least that is what I guess, and it could be that with hyperthreading on, you get more reliable results, because the operating system has more manoeuvering room, it could leave the four matches alone and schedule other background processess on the hyperthreads. I am no programmer, but I think you could test this well enough? If you don't want hyperthreading why not invest in another i5, that would amount to the same thing as an i7 with hyperthreading off. But why would hyperthreading necessarily be worse. By the way Marco Costalba posted the following:

mcostalba commented on 37e9802 7 hours ago

Thanks Jean-Francois,

unfortunately at longer TC results are not so good (see my last
patch log message).

If I may comment on your test setup, I'd suggest to avoid …
-concurrency 4
and always play one match at a time. I have verified in the past that
even -concurrency 2 is unreliable and results are not reproducible.

Eelco

[hgm]

OK, thanks everyone for the helpful info. Indeed it was possible to solve the poblem without switching off hyper-theading, with the use of affinity. First I made a little test program that (in an infinite loop) counts to a billion, and then prints how long that took. If it is the only program that runs, or if it runs twice on different physical cores, it prints a stream of "0.566 sec". When I set the affinity of two of them to the same virtual core, this doubles (as expected, because the OS now refuses to run both at the same time, and divides the time slices of that HT between them). Interesting thing is that the reported CPU usage in the task manager then drops from 13% to 6%. If I run two on different virtual cores that map onto the same physical core, they slow down to ~0.780 sec.

This way I was able to establish that the core numbering, as used in the affinity popup of the task manager, has 0 and 1 on physical core 1, 2 and 3 on physical core 2, etc. So whatI do now is that I set affinities of my first WinBoard to 0&1, of the second to 2&3 etc. The engines indeed inherit this, as I can check in the task manager. This way different matches can never steal CPU from each other. As I test with ponder off, one physical core should be enough for each match. The GUI can make use of the second hyper thead for the little time it consumes while an engine is running (handling thinking output) to have minimal impact on the engine performance. (No need for the OS to temporarily suspend the pocess of the running engine when the GUI process requests CPU; it simply schedules the GUI on the idle HT.)

I hope this method of testing will cure the erratic and contradictory results I was getting. My paranoic mind can still imagine artifacts by unfavoable mapping of the various processes into the shared L3 cache: if two programs happen to collide there, they might see a significant peformance degradation, constantly flushing each other's code and data out of the caches. I guess there is no way to control this.

[Laskos]

The model is Acer Aspire M3970.

The bios on the Acer Aspire M3970 has not an option to disable HT, sadly.

But you don't need it. Run the exe (I presume winboard.exe) with affinities, to use only physical CPUs and you will be fine.

For example, for 4 cores, 8 threads, start a command prompt inside the folder of winboard and type:

Start /affinity 55 winboard.exe

55 is the hexadecimal representation of 01010101 (from core 7 to 0), i.e. core 0, 2, 4, 6

The children the program (winboard) spawns (in this case the exe of the engines) willl automatically inherit the same affinities.

I got an i7 2600 and have a similar problem with HT. In LittleBlitzer, without setting affinities I have

Code: Select all

 1.  Houdini 3 4-threads      	9.0/22	1-5-16  	(L: m=5 t=0 i=0 a=0)	(D: r=16 i=0 f=0 s=0 a=0)	(tpm=978.8 d=22.36 nps=9374975)
 2.  Houdini 3 8-threads      	13.0/22	5-1-16  	(L: m=1 t=0 i=0 a=0)	(D: r=16 i=0 f=0 s=0 a=0)	(tpm=985.6 d=22.64 nps=11659571)

When 4-threaded Houdini is running, CPU load is at 50%, 8-threaded 100%. 8-threaded seems to be 25% faster, searches a bit deeper, and seems to be stronger than 4-threaded (not sure). Does it has something to do with Turbo Boost going from 3.4 to 3.8GHz?

When I set
Start /affinity 55 LittleBlitzer.exe
I get something ridiculous

Code: Select all

 1.  Houdini 3 4-threads      	8.5/10	7-0-3  	(L: m=0 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=939.3 d=26.05 nps=10245732)
 2.  Houdini 3 8-threads      	1.5/10	0-7-3  	(L: m=7 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=1087.1 d=15.46 nps=369459)

Can I make a 4-threaded Houdini run at full CPU load?

[yanquis1972]

i have been using 8 (sometimes 7) threads lately as well. i do immediately notice a good jump in kn/s, but haven't done any time to depth comparison, which would be the true indicator. it's definitely much more intensive on my system than 4 (too much so if i don't lower priority). i don't know if its an artificial inflation, but if it is, the count can't be multipled by # of detected 'cores' as it's only some percentage higher (not anywhere close to 80%)...wonder if robert could chime in on that one. i've been under the impression that the only engine said (by harvey williamson, have not heard it directly from the authors mouth) to gain from HT was HIARCS.

edit -- 4 threads with HT enabled should show as 50% at full load. this is completely expected. also, turbo boost will only 'supercharge' one core, if the others are idle.