How can I switch off hyperthreading?

[Lavir]

When I set
Start /affinity 55 LittleBlitzer.exe
I get something ridiculous

Code: Select all

 1.  Houdini 3 4-threads      	8.5/10	7-0-3  	(L: m=0 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=939.3 d=26.05 nps=10245732)
 2.  Houdini 3 8-threads      	1.5/10	0-7-3  	(L: m=7 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=1087.1 d=15.46 nps=369459)

Can I make a 4-threaded Houdini run at full CPU load?

It's obvious you get those results. You forced windows to use only physical cores with set affinity, but in one instance you force Houdini to use the full 8 threads (that it will try to use since it still see also the logical cores because they are still there, also if unusable given the affinity) so that it actually divides two threads on a single core (causing a lot of slowdown).

So it's obvious that the version with 8 threads used will lose a lot, since it is very slowed down.

As for hyperthreading: you usually get higher nodes (about a 20%-30% increase in total), yes, but it is usually said that for chess it is better 4 x 2.000 than 8 x 1.100, for example; in nodes the second results seems better (8.800 vs 8.000) but in practice many experts said it is worse. The technical reason of why that's so a chess programmer I think can explain it better than I can (in the Houdini faq for example Robert say it has to do with inefficiency in the parallel alpha-beta search).

[Laskos]

When I set
Start /affinity 55 LittleBlitzer.exe
I get something ridiculous

Code: Select all

 1.  Houdini 3 4-threads      	8.5/10	7-0-3  	(L: m=0 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=939.3 d=26.05 nps=10245732)
 2.  Houdini 3 8-threads      	1.5/10	0-7-3  	(L: m=7 t=0 i=0 a=0)	(D: r=3 i=0 f=0 s=0 a=0)	(tpm=1087.1 d=15.46 nps=369459)

Can I make a 4-threaded Houdini run at full CPU load?

It's obvious you get those results. You forced windows to use only physical cores with set affinity, but in one instance you force Houdini to use the full 8 threads (that it will try to use since it still see also the logical cores because they are still there, also if unusable given the affinity) so that it actually divides two threads on a single core (causing a lot of slowdown).

So it's obvious that the version with 8 threads used will lose a lot, since it is very slowed down.

As for hyperthreading: you usually get higher nodes (about a 20%-30% increase in total), yes, but it is usually said that for chess it is better 4 x 2.000 than 8 x 1.100, for example; in nodes the second results seems better (8.800 vs 8.000) but in practice many experts said it is worse. The technical reason of why that's so a chess programmer I think can explain it better than I can (in the Houdini faq for example Robert say it has to do with inefficiency in the parallel alpha-beta search).

Ok, thanks. I am running a test match, for now 8-threaded and 4-threaded are pretty even (without setting affinities). Using only the 4-threaded engine, is it better to set affinity 55 than setting no affinity at all?

[Lavir]

Using only the 4-threaded engine, is it better to set affinity 55 than setting no affinity at all?

With affinity 55 you force Windows to use only the 4 physical cores of the 8 threads. Generally speaking Windows (or any recent OS) should automatically give preference to physical cores, but setting affinities is a nice way to be certain that only physical cores are used at all times (and leave logical cores for the GUI or other things).

[yanquis1972]

could someone explain how one would set affinities for eg chessbase GUI (if possible)? i've known this is best as vas/lukas said as much on rybka forum, but i thought it was only possible w/ rybka.

[Laskos]

could someone explain how one would set affinities for eg chessbase GUI (if possible)? i've known this is best as vas/lukas said as much on rybka forum, but i thought it was only possible w/ rybka.

My understanding, as Fabio explained, is that I guess you have to use
start /affinity 55 ChessProgram13.exe
This is in order to use only 4 physical cores of the 8 threads.

I ran the Fritz benchmark with interesting results:
Benchmarks without affinities set (multiple tries):
8 threads 28
4 threads 21

With affinity 55 (4 physical cores):
4 threads 22.5

Setting affinity improves 4 thread performance by some maybe 7%, probably depending on what other services are running

Kai

[Laskos]

Curious results running Houdini 3 Tactical on Arasan 15 test suite (5s position) on my i7 2600.

4 threads: 161/200 Average time 1.29s
8 threads: 167/200 Average time 1.57s

Maybe, after all, HT is beneficial?

[shrapnel]

Curious results running Houdini 3 Tactical on Arasan 15 test suite (5s position) on my i7 2600.

4 threads: 161/200 Average time 1.29s
8 threads: 167/200 Average time 1.57s

Maybe, after all, HT is beneficial?

YES, it is !
This is what I've been saying for some time.
Read this thread for further details http://www.talkchess.com/forum/viewtopic.php?t=45863.
To summarize, if you can get 20 % or more kN/s with Hyper-threading, in my experience, there is a DEFINITE increase in the ELO of the Program !
For instance, on my 6-core i7 3930 k overclocked to 4.8 Ghz, with HT disabled, I get 20,565 kN/s (Fritz Benchmark) and with all 12 threads enabled I get more than 26,000 kN/s !!....much more than a 20 % increase !
I regularly play online Engine-Engine matches against others on www.come2play.com ; I have found that I'm now able to regularly beat other strong Engine players,against whom I could only get draws earlier !
In game, my kN/s speed goes much over 27000-28000 kN/s speed..... . I have NEVER felt that Houdini has become weaker by enabling HT, only much, much STRONGER !!!

[Houdini]

Curious results running Houdini 3 Tactical on Arasan 15 test suite (5s position) on my i7 2600.

4 threads: 161/200 Average time 1.29s
8 threads: 167/200 Average time 1.57s

Maybe, after all, HT is beneficial?

I'm surprised that an experienced tester like yourself believes that this difference is significant.
Note also that the average time is a more relevant statistic than the number of solutions found.

Robert

[Houdini]

To summarize, if you can get 20 % or more kN/s with Hyper-threading, in my experience, there is a DEFINITE increase in the ELO of the Program !

Correct, a speed increase of more than 20% is needed to overcome the 20% efficiency drop from using twice as many threads.

For instance, on my 6-core i7 3930 k overclocked to 4.8 Ghz, with HT disabled, I get 20,565 kN/s (Fritz Benchmark) and with all 12 threads enabled I get more than 26,000 kN/s !!....much more than a 20 % increase !

The Fritz Benchmark is a poor performance indicator for Houdini.
What are your Houdini 3 "autotune" speeds with 4 and 8 threads?

I regularly play online Engine-Engine matches against others on www.come2play.com ; I have found that I'm now able to regularly beat other strong Engine players,against whom I could only get draws earlier !
In game, my kN/s speed goes much over 27000-28000 kN/s speed..... . I have NEVER felt that Houdini has become weaker by enabling HT, only much, much STRONGER !!!

The strength increase would be at most about 10 Elo, it would be difficult to see the difference unless you play many hundreds of games.
Your "much much STRONGER" can only be a placebo effect.

Robert

[IQ]

Does Houdini 3 still have the same bug as Houdini 2 with affinities set to physical cores (0,2,4 etc)?

When you set affinities in Houndini sometimes the threads are not created on different cores (using 6 threads on a 6 core 12 thread machine). Instead two worker threads are stuck on the same physical core (with one core idling) this slows down Houdini tremendously (sometimes 40% and sometimes even more). This is an awkward bug as it is relatively rare (once or twice a game) and also it can get unstuck after some moves. One can also provoke this in analysis mode by quickly going to new positions - usually after hopping around in a game 20 times one thread gets misallocated (for example two worker threads on core 2 and core 4 is idle). This only happens with afinities set. Without setting affinities and still using 6 threads on a 6 core 12 thread machine it seems not to happen or very very very rarely (i recall only one such incident). Off course with HT off in bios everything works fine.