How can I switch off hyperthreading?

[Albert Silver]

The Fritz Benchmark is a poor performance indicator for Houdini.
What are your Houdini 3 "autotune" speeds with 4 and 8 threads?

Hi Mr Houdart, I'm a big fan of yours, so I don't wish to argue with you too much . I wonder why you want "autotune" speeds with 4 and 8 threads as I'm running an i7 3930 k. But anyway, here you go...
This with 8 threads :-


Hmm..... a 48 % increase with 8 threads as compared to 4 threads.... seems that you were right about the Fritz benchmark being a poor indicator.... only, 'autotune' results only seem to reinforce the point I was trying to make

I hate to tell you this, but the NPS is a useless way to measure it.

[Laskos]

The Fritz Benchmark is a poor performance indicator for Houdini.
What are your Houdini 3 "autotune" speeds with 4 and 8 threads?

Hi Mr Houdart, I'm a big fan of yours, so I don't wish to argue with you too much . I wonder why you want "autotune" speeds with 4 and 8 threads as I'm running an i7 3930 k. But anyway, here you go...
This with 8 threads :-


Hmm..... a 48 % increase with 8 threads as compared to 4 threads.... seems that you were right about the Fritz benchmark being a poor indicator.... only, 'autotune' results only seem to reinforce the point I was trying to make

I hate to tell you this, but the NPS is a useless way to measure it.

Practically very little separates in strength 8 threads from 4 threads with HT on on 4 cores

Code: Select all

PC, Blitz 1m+1s 

Houdini 3 Pro x64 4 threads    +2  +90/=162/-88 50.29%  171.0/340
   
Houdini 3 Pro x64 8 threads    -2  +88/=162/-90 49.71%  169.0/340

But using only 4 threads is more convenient, as there is space to accomodote other running services and the comp is much more responsive.

[kasinp]

Does Houdini 3 still have the same bug as Houdini 2 with affinities set to physical cores (0,2,4 etc)?

When you set affinities in Houndini sometimes the threads are not created on different cores (using 6 threads on a 6 core 12 thread machine). Instead two worker threads are stuck on the same physical core (with one core idling) this slows down Houdini tremendously (sometimes 40% and sometimes even more). This is an awkward bug as it is relatively rare (once or twice a game) and also it can get unstuck after some moves. One can also provoke this in analysis mode by quickly going to new positions - usually after hopping around in a game 20 times one thread gets misallocated (for example two worker threads on core 2 and core 4 is idle). This only happens with afinities set. Without setting affinities and still using 6 threads on a 6 core 12 thread machine it seems not to happen or very very very rarely (i recall only one such incident). Off course with HT off in bios everything works fine.

Unless I don't understand what you're saying, what you describe is a Windows scheduler bug.
If you *force* process affinities in Windows, how could Houdini (or any other process) ever circumvent this?

Robert

Let me explain again. System: a 990X with HT on (6 real cores 12 HThreads), LP in use and affinities set to the even cores (0,2,4,6,8,10). Houdini will sometimes not use all even cores. Very rarely it will get stuck and use only 5 (sometimes even less cores). As Houdini is set for 6 Threads this means that two of houdinis threads are allocated to the same physical core. This slows Houdini down (sometimes 40%, and if more threads are glued to the same core even worse). This is rare! But one can provoke it. I tried it on several I7 processors on different machine - on the 4 core (8HT) is was extremly difficult to reproduce - so much that it is a non issue. But on the 6 core (12HT) machines it happens much more frequently. Without setting affinities this did not happen - the load is distributed evenly between the 12hthreads (or Houdnini recovers much quicker and unnoticably). So if you have a 6 physical core+ machine, i would advise to turn HT off, if you only want to use 6 threads.

P.S.: It might not be a Windows 7 scheduler error. I believe it could also be a race condition in the houdini multithreaded code.

I can confirm exactly the same symptoms on my 6/12 core machine.
BTW, turning off hyperthreading has the added effect of saving energy.
Temperatures reported by PC Probe are in the range of 6-8C less with HT off - interestingly, the comparison here is between 6 cores with HT off and 6 cores with 6 HT cores "idle". Idle or not - they still *always* add heat.

PK

[Albert Silver]

The Fritz Benchmark is a poor performance indicator for Houdini.
What are your Houdini 3 "autotune" speeds with 4 and 8 threads?

Hi Mr Houdart, I'm a big fan of yours, so I don't wish to argue with you too much . I wonder why you want "autotune" speeds with 4 and 8 threads as I'm running an i7 3930 k. But anyway, here you go...
This with 8 threads :-


Hmm..... a 48 % increase with 8 threads as compared to 4 threads.... seems that you were right about the Fritz benchmark being a poor indicator.... only, 'autotune' results only seem to reinforce the point I was trying to make

I hate to tell you this, but the NPS is a useless way to measure it.

Practically very little separates in strength 8 threads from 4 threads with HT on on 4 cores

Code: Select all

PC, Blitz 1m+1s 

Houdini 3 Pro x64 4 threads    +2  +90/=162/-88 50.29%  171.0/340
   
Houdini 3 Pro x64 8 threads    -2  +88/=162/-90 49.71%  169.0/340

But using only 4 threads is more convenient, as there is space to accomodote other running services and the comp is much more responsive.

Sedat brought up very good data on this a while back. The NPS reported is completely pointless and is not an indicator in any way of strength one way or the other. HT loses strength with chess engines simply due to the inefficiency of added threads. Bob Hyatt explained very clearly this and the reasons for it.

[shrapnel]

The NPS reported is completely pointless and is not an indicator in any way of strength one way or the other. HT loses strength with chess engines simply due to the inefficiency of added threads.

You guys simply can't seem to see the woods for the trees ! Even Robert Houdart admits that the ELO will increase if one can achieve more than a 20% increase in the kN/s with HT on. The keywords are more than 20 %...get it ? So, instead of mindlessly repeating like parrots" HT loses strength with Chess engines" try to open your closed minds a little and and at least consider the possibility that ELO may NOT decrease ! (No offence).
On another note, I recently played a forum member kgburcham who runs dual-Xeons (12 cores) on www.come2play.com. Even using his 12 physical 'real' cores AND with the White pieces, he was barely able to hold me ( with HT on and H3 running on all 12 threads, i7 3930 k o/c to 4.8 Ghz) to a draw !
I'm almost certain I can beat him next time, with the White pieces. His userid was Houdini Program.
My userid there is sh1va..... anyone who thinks that HT is bad is welcome to challenge me. I will wipe the floor with them, that too with HT on ! If I lose, I will delete my userid there !

[Houdini]

Hello Anil, I'm still interested in seeing your 6 and 12 thread "autotune" results. Do you obtain a speed increase larger than 20%?

[kgburcham]

On another note, I recently played a forum member kgburcham who runs dual-Xeons (12 cores) on www.come2play.com.
This is not a true statement

Even using his 12 physical 'real' cores AND with the White pieces, he was barely able to hold me to a draw!
This is not a true statement

I'm almost certain I can beat him next time,
with the White pieces.
This is a true statement

His userid was Houdini Program.
This is not a true statement

I will wipe the floor with them,
This is not a true statement if the opponent is using Houdini 3 with a good book

If I lose, I will delete my userid there !
This is a true statement

kgburcham

[Laskos]

Hello Anil, I'm still interested in seeing your 6 and 12 thread "autotune" results. Do you obtain a speed increase larger than 20%?

I am new to this stuff with my new and cheap i7 2600 (a bit overclocked), but FWIW here in short are my results:
Optimal Split Depth was 12.
128MB Hash: 36% increase in speed from 4 to 8 threads
4096MB Hash: 28% increase

An observation: with 8 threads, CPU kicks in slowly, the first second-two it is as slow as with 4 threads (often slower), and only after 5-7 seconds the speed increases significantly compared to 4 threads. Maybe that's the reason on Arasan test suite the time to solution was higher, most of the problems were solved in less than 2 seconds.

Did someone reliably tested on longer TC than bullet the benefit/harm of HT?

Kai

[Houdini]

I am new to this stuff with my new and cheap i7 2600 (a bit overclocked), but FWIW here in short are my results:
Optimal Split Depth was 12.
128MB Hash: 36% increase in speed from 4 to 8 threads
4096MB Hash: 28% increase

With these numbers the HT becomes beneficial.
It seems like Intel is steadily improving the efficiency of the hyper-threading.

Can you make the same comparison with Houdini 2.0? Thank you.

Robert

[Laskos]

I am new to this stuff with my new and cheap i7 2600 (a bit overclocked), but FWIW here in short are my results:
Optimal Split Depth was 12.
128MB Hash: 36% increase in speed from 4 to 8 threads
4096MB Hash: 28% increase

With these numbers the HT becomes beneficial.
It seems like Intel is steadily improving the efficiency of the hyper-threading.

Can you make the same comparison with Houdini 2.0? Thank you.

Robert

Strange, tested only with 128MB Hash, increase in peak (for different split depths) of only 8% for Houdini 2. Seems something happened with Houdini 3 HT/non-HT (36%).

Kai