Is hyper-threading worse for chess programs?

[reflectionofpower]

I am curious because of what Robert states in his manual on his website:

"Q: I'm running Houdini on a Core i7 CPU with hyper-threading. Would you recommend to use hyper-threading with Houdini?

The architecture of Houdini (and of chess engines in general) is not very well suited for hyper-threading; using more threads than physical cores will usually degrade the performance of the engine. Although the hyper-threads often produce a slightly higher node speed, the increased inefficiency of the parallel alpha-beta search more than offsets the speed gain obtained with the additional hyper-threads.
To give a practical example, it's more efficient to use 4 threads running at 2,000 kN/s each than 8 threads running at 1,100 kN/s each, although the latter situation produces a higher total node speed.
For this reason it's best to set the number of threads not higher than the number of physical cores of your hardware."

I have noticed most programs get a performance gain of 25% in node count but does it really hurt performance overall by being less efficient because of the hyper-threading?

[yanquis1972]

there was some discussion about this awhile back. as i recall it was ultimately shown that, with H3 at least, HT might in fact be advantageous. again as i recall, robert was open to the possibility, though he did not expect the results demonstrated.

i stick with 4CPU because the load is much lighter (another reason i think 8 CPU might in fact be optimal) & makes multitasking possible, while still getting solid engine performance.

[Mike S.]

I can only speak for the Intel i5-3210M dual core cpu. I clearly get gains from hyperthreading in terms of time to depth, node rate, and also the solution numbers in a test suite are bigger.

Code: Select all

Intel i5-3210M, 2 x 2.5 GHz (-2.9 GHz)
2 physical cores; 4 logical cores (HT)
512 M hash tables (DDR3-RAM / 800 MHz)

Engine           | P#1 depth  time(2T)  time(4T) | P#2 depth  time(2T)  time(4T)
--------------------------------------------------------------------------------
Critter 1.6a     |      20      34        20     |      19      59        34
Deep Fritz 13    |      21      48        62 ?   |      22      32        15
Houdini 1.5a     |      21      40        26     |      21     113        36 !
Rybka 2.3.2a     |      16      36        26     |      16     113       136 ?
Stockfish 100413 |      24      34        13 !   |      25      66        41
--------------------------------------------------------------------------------
+UPDATE:

Engine           | P#3 depth  time(2T)  time(4T) | P#4 depth  time(2T)  time(4T)
--------------------------------------------------------------------------------
Crafty 23.04 cbn.|      21      90        49     |      23      46        46 ?
Shredder Cl. 2012|      14      32        35 ?   |      16      60        36
Spark 1.0        |      19      55        17 !   |      22      45        30
--------------------------------------------------------------------------------
Average Time Relation total:      ~1.73:1
--------------------------------------------------------------------------------
AvTR. with (?)+(!) excluded:      ~1.68:1
------------------------------------------

positions with FEN see below

#1 r1bq1rk1/2ppbppp/p1n2n2/1p2p3/4P3/1B3N2/PPPP1PPP/RNBQR1K1 w - -
#2 5k2/6p1/2p2p2/P7/1Q6/2P1pqPP/7K/8 b - - bm c5; id Quick-19;
#3 starting position
#4 r5k1/p2r1bpp/2p2p2/8/n1P5/P5B1/5PPP/2R1RBK1 w - - bm c5; id "Mike's Test 2.2, Nr. 26";

Vergleich der kN/s: Pos. #1, exakt 0:30 pro Zug

Engine              2T    4T  Faktor
------------------------------------
Critter 1.6a       2667  3930  1,47
Houdini 1.5a       3408  3964  1,16
Rybka 2.3.2a        168   283  1,68
Shredder Cl. 2012   626  1404  2,24
Spark 1.0          4304  5644  1,31
                              ------
                            Ø  1,57
------------------------------------

              Houdini 1.5a 4T	 Stockfish 100413 4T  Critter 1.6a 4T  Stockfish 100413 2T  Houdini 1.5a 2T  Critter 1.6a 2T	
----------------------------------------------------------------------------------------------------------------------------
Total time:   00:02:23           00:03:11             00:03:46         00:03:18             00:03:42         00:04:04	
Total solved: 56                 55                   54               53                   50               47	

See also http://www.talkchess.com/forum/viewtopi ... =0&t=47757 with a huge discussion about this topic.

[geots]

I am curious because of what Robert states in his manual on his website:

"Q: I'm running Houdini on a Core i7 CPU with hyper-threading. Would you recommend to use hyper-threading with Houdini?

The architecture of Houdini (and of chess engines in general) is not very well suited for hyper-threading; using more threads than physical cores will usually degrade the performance of the engine. Although the hyper-threads often produce a slightly higher node speed, the increased inefficiency of the parallel alpha-beta search more than offsets the speed gain obtained with the additional hyper-threads.
To give a practical example, it's more efficient to use 4 threads running at 2,000 kN/s each than 8 threads running at 1,100 kN/s each, although the latter situation produces a higher total node speed.
For this reason it's best to set the number of threads not higher than the number of physical cores of your hardware."

I have noticed most programs get a performance gain of 25% in node count but does it really hurt performance overall by being less efficient because of the hyper-threading?

I likely will never know. When I took my new Alienware AURORA R4 Intel i7 6-core system out of the box- the first thing I did after plugging it into the power supply was go straight to the bios AND DISABLE hyperthreading- for eternity. I did not even stop at "Go" to collect $200.

gts

[shrapnel]

I am curious because of what Robert states in his manual on his website:

"Q: I'm running Houdini on a Core i7 CPU with hyper-threading. Would you recommend to use hyper-threading with Houdini?

The architecture of Houdini (and of chess engines in general) is not very well suited for hyper-threading; using more threads than physical cores will usually degrade the performance of the engine. Although the hyper-threads often produce a slightly higher node speed, the increased inefficiency of the parallel alpha-beta search more than offsets the speed gain obtained with the additional hyper-threads.
To give a practical example, it's more efficient to use 4 threads running at 2,000 kN/s each than 8 threads running at 1,100 kN/s each, although the latter situation produces a higher total node speed.
For this reason it's best to set the number of threads not higher than the number of physical cores of your hardware."

I have noticed most programs get a performance gain of 25% in node count but does it really hurt performance overall by being less efficient because of the hyper-threading?

I likely will never know. When I took my new Alienware AURORA R4 Intel i7 6-core system out of the box- the first thing I did after plugging it into the power supply was go straight to the bios AND DISABLE hyperthreading- for eternity. I did not even stop at "Go" to collect $200.

gts

Maybe you are right. In the previous discussion, I was of the opinion that HT IS beneficial.
Certainly, like the OP mentions, the node count increases and time to reach a certain Depth decreases.
But of late, I've changed my mind about this and now believe that HT is actually detrimental to the performance of Houdini.
It is my belief that though there is no perceptible fall in performance, Houdini gets held to draws more often. I believe that when HT is enabled,sometimes, somewhere in deep analyses, small errors creep in. These slight errors are not obvious and don't necessarily lead to a loss, but many Draws are snatched from the jaws of victory, so to speak.
So, its back to HT off for me.

[geots]

I am curious because of what Robert states in his manual on his website:

"Q: I'm running Houdini on a Core i7 CPU with hyper-threading. Would you recommend to use hyper-threading with Houdini?

The architecture of Houdini (and of chess engines in general) is not very well suited for hyper-threading; using more threads than physical cores will usually degrade the performance of the engine. Although the hyper-threads often produce a slightly higher node speed, the increased inefficiency of the parallel alpha-beta search more than offsets the speed gain obtained with the additional hyper-threads.
To give a practical example, it's more efficient to use 4 threads running at 2,000 kN/s each than 8 threads running at 1,100 kN/s each, although the latter situation produces a higher total node speed.
For this reason it's best to set the number of threads not higher than the number of physical cores of your hardware."

I have noticed most programs get a performance gain of 25% in node count but does it really hurt performance overall by being less efficient because of the hyper-threading?

I likely will never know. When I took my new Alienware AURORA R4 Intel i7 6-core system out of the box- the first thing I did after plugging it into the power supply was go straight to the bios AND DISABLE hyperthreading- for eternity. I did not even stop at "Go" to collect $200.

gts

Maybe you are right. In the previous discussion, I was of the opinion that HT IS beneficial.
Certainly, like the OP mentions, the node count increases and time to reach a certain Depth decreases.
But of late, I've changed my mind about this and now believe that HT is actually detrimental to the performance of Houdini.
It is my belief that though there is no perceptible fall in performance, Houdini gets held to draws more often. I believe that when HT is enabled,sometimes, somewhere in deep analyses, small errors creep in. These slight errors are not obvious and don't necessarily lead to a loss, but many Draws are snatched from the jaws of victory, so to speak.
So, its back to HT off for me.

Think of it this way. You know for 100% sure that having HT OFF will not cause you any problems. But with HT ON- you are like me. You can't be sure. So take the safe route.



Best,

george

[syzygy]

Think of it this way. You know for 100% sure that having HT OFF will not cause you any problems. But with HT ON- you are like me. You can't be sure. So take the safe route.

Do what you wish, but disabling HT in the BIOS is bad advice.

There is no need to disable HT in the BIOS (unless you run WIndows 3.11 or so).

Regarding the number of threads to use, there are many arguments in favour of setting it equal to the number of physical cores:
- if there is at all a benefit to using all hyperthreads, it is small.
- using all hyperthreads will make your system less responsive.
- using all hyperthreads consumes more energy.
- using all hyperthreads will likely degrade performance on a system that has other tasks running (and it might even degrade performance if no other tasks are running).

This is just general advice, everybody should do what he likes best. But there is no point in disabling HT in the BIOS.

[jpqy]

You forget one Roland..

With HT Off you can overclock your system higher with a nice speed gain!
Difference with HT ON = +/-10°C

Kind regards,
Jean-Paul.

[syzygy]

With HT Off you can overclock your system higher with a nice speed gain!
Difference with HT ON = +/-10°C

Only because you don't have all threads running anymore...

This is more an argument in favour of setting the number of threads used by an engine equal to the number of physical cores than to disable HT in the BIOS.

[geots]

With HT Off you can overclock your system higher with a nice speed gain!
Difference with HT ON = +/-10°C

Only because you don't have all threads running anymore...

This is more an argument in favour of setting the number of threads used by an engine equal to the number of physical cores than to disable HT in the BIOS.

Ok, I won't disable it in the bios, IF you can show me an easier way to disable it.


gts