TNGCC - need to decide ponder ON or OFF (new poll)

[Vinvin]

Why not : "3 threads, ponder OFF, ht ON" or "4 threads, ponder OFF, ht ON" ?
I'd vote for one of these ...

Actually, I am leaning to the latter as per Robert Houdart's post in the other thread.

I am just going to do some testing.

Here are some config and numbers that can help you : http://www.talkchess.com/forum/viewtopi ... 887#437887

[Martin Thoresen]

Here are some config and numbers that can help you : http://www.talkchess.com/forum/viewtopi ... 887#437887

Hmm, those are not the results I would expect. The ones relevant for me are these:

1) Houdini 2.0c HT OFF 6 physical cores

44s
21s
44s
25s
42s
31s
42s
46s
27s
59s
---
Note:
Houdini 2.0c HT OFF 6 Cores solves the mate average in 38s

4) Houdini 2.0c HT ON 6 Threads (50 % CPUs usage)


59s
46s
65s
29s
38s
19s
44s
68s
62s
25s
---
Some Notes:
Houdini 2.0c HT ON 6 Threads solves the mate average in 45s
Houdini 2.0c 6T has been tested with 6 CPUs,i mean rest 6 CPUs were on idle
In other words:Houdini's during thinking process,there was no any other engine

When using the same number of threads, and having HT activated yields a loss of performance in this test. How to explain that?

Houdini 2.0c HT OFF 6 physical cores: 38s
Houdini 2.0c HT ON 6 Threads (50 % CPUs usage): 45s

Then I don't get why Robert Houdart suggested that I keep 4 threads but HT on...? The engine doesn't get the 4 real cores, but a mix between that and the hyper-threads or what?




My own little HT test, 4 vs 7 threads for Houdini and Stockfish:

HOUDINI 3
4 threads, ht ON
time 11297 nodes 112075387 nps 9920000
time 12380 nodes 122268853 nps 9876000
time 7510 nodes 73601648 nps 9800000
time 12958 nodes 130564744 nps 10075000
time 13988 nodes 138448661 nps 9897000
AVG
time 11627 nodes - nps 9913600

7 threads, ht ON
time 8945 nodes 107650169 nps 12034000
time 10228 nodes 125351190 nps 12255000
time 6120 nodes 72286199 nps 11811000
time 8805 nodes 106799868 nps 12129000
time 7905 nodes 94139539 nps 11908000
AVG
time 8401 nodes - nps 12027400

STOCKFISH 2.3.1
4 threads, ht ON, bench 512 4 16 default depth
==========================
Total time (ms) : 6366
Nodes searched : 42613141
Nodes/second : 6693864
==========================
Total time (ms) : 8067
Nodes searched : 54665591
Nodes/second : 6776446
==========================
Total time (ms) : 5914
Nodes searched : 39650594
Nodes/second : 6704530
==========================
Total time (ms) : 7099
Nodes searched : 47720033
Nodes/second : 6722078
==========================
Total time (ms) : 6053
Nodes searched : 40656345
Nodes/second : 6716726

AVG
Total time (ms) : 6700
Nodes searched : -
Nodes/second : 6722789


7 threads, ht ON, bench 512 7 16 default depth
==========================
Total time (ms) : 6943
Nodes searched : 46456963
Nodes/second : 6691194
==========================
Total time (ms) : 5929
Nodes searched : 38081955
Nodes/second : 6422997
==========================
Total time (ms) : 6975
Nodes searched : 46551260
Nodes/second : 6674015
==========================
Total time (ms) : 5851
Nodes searched : 39036964
Nodes/second : 6671844
===========================
Total time (ms) : 6849
Nodes searched : 45615208
Nodes/second : 6660126

AVG
Total time (ms) : 6509
Nodes searched : -
Nodes/second : 6624035

[Lavir]

But also that I have actually done these tests, and that no amount of talk could ever convince me that what I have seen with my own eyes would not be true...

You say these things but then you don't provide the types of testings you have done.

Watch for example the testing in the other thread of the same Martin when using asymmetrical logical cores and you will see in practice of what I talk about. You will immediately see that there's something odd in the numbers and that using logical cores in an asymmetric manner compromise the results.

Which testings have you done that should prove the contrary?

So perhaps we better drop it. People can decide for themselves whether they will believe you or me.

Fine. I agree that's futile to continue because what we had to say has already been said and people can come to their own conclusions.

[Lavir]

Then I don't get why Robert Houdart suggested that I keep 4 threads but HT on...? The engine doesn't get the 4 real cores, but a mix between that and the hyper-threads or what?

IIRC those tests where done with Windows XP, and that was a great factor in those results. XP was very bad on threads management and so you had these types of problems (and for this all the testings done by Sedat are wrong). XP didn't rightly assign physical processes to applications that made constant use of the CPU, but instead mixed cores with the threads.

Windows Vista and especially 7/8 have greatly improved thread management and this doesn't happen.

With the latest Windows using 4 cores with HT ON is the same as using them with HT OFF, since the scheduler will rightly assign physical cores to the engines and relegate the logical for the other tasks (assuming naturally you don't start another process that require high CPU resources and so it will be switched to a physical core, especially if the engines are run at lower priorities). It is even a little tiny beneficial since the GUI etc. will not steal cycles from the engines.

Anyway if you want to be 100% sure you can just set affinities to force engines to use only the physical cores. It is very easy to do, since just setting them on the GUI will inherit them on the engines automatically. As I said it is not needed if you have Windows Vista or later, but if you want to be always 100% sure that engines are forced to use only physical cores that's the way.

[Vinvin]

Then I don't get why Robert Houdart suggested that I keep 4 threads but HT on...? The engine doesn't get the 4 real cores, but a mix between that and the hyper-threads or what?

IIRC those tests where done with Windows XP, and that was a great factor in those results. XP was very bad on threads management and so you had these types of problems (and for this all the testings done by Sedat are wrong). XP didn't rightly assign physical processes to applications that made constant use of the CPU, but instead mixed cores with the threads.

Windows Vista and especially 7/8 have greatly improved thread management and this doesn't happen.

With the latest Windows using 4 cores with HT ON is the same as using them with HT OFF, since the scheduler will rightly assign physical cores to the engines and relegate the logical for the other tasks (assuming naturally you don't start another process that require high CPU resources and so it will be switched to a physical core, especially if the engines are run at lower priorities). It is even a little tiny beneficial since the GUI etc. will not steal cycles from the engines.

No, the OS config was "Windows Vista Ultimate 64-bit", as far I know it's the same threads management as Win 7.

[Lavir]

No, the OS config was "Windows Vista Ultimate 64-bit", as far I know it's the same threads management as Win 7.

Then something strange DID happen (probably some background service/process) because this never happens with recent Windows. Sadly OS configuration is a very important thing and many odd things can happen if it's not configured properly.

Anyway as I said previously you can just force the OS to have the GUI/engines use only the physical cores to be sure.

As an example, if you use Fritz 13 and have 4 cores/8 threads, just open a command prompt where the exe is and write:

start /affinity 55 chessprogram13.exe

Nothing else is required. Engines will automatically inherit the correct affinities. Fritz 13 has also an automatic "smart CPU managements" that put the cores to 4 (if you have HT ON with 8 threads).

[Vinvin]

Here are some config and numbers that can help you : http://www.talkchess.com/forum/viewtopi ... 887#437887

Hmm, those are not the results I would expect. The ones relevant for me are these:

1) Houdini 2.0c HT OFF 6 physical cores

...
---
Note:
Houdini 2.0c HT OFF 6 Cores solves the mate average in 38s

4) Houdini 2.0c HT ON 6 Threads (50 % CPUs usage)


...
---
Some Notes:
Houdini 2.0c HT ON 6 Threads solves the mate average in 45s
Houdini 2.0c 6T has been tested with 6 CPUs,i mean rest 6 CPUs were on idle
In other words:Houdini's during thinking process,there was no any other engine

When using the same number of threads, and having HT activated yields a loss of performance in this test. How to explain that?

Some math give : "HT on" give +50% raw computation power. But here, it's divide by 2 because other half is use by another engine so, 150%/2 = 75% speed : 38 sec/0.75 = 50 sec ; about the 45 sec, I suppose the statistical error is the explanation between 50 and 45.

[hgm]

When using the same number of threads, and having HT activated yields a loss of performance in this test. How to explain that?

Isn't that expected? With HT on your CPU looks like 12 logical cores, each with a capacity of about 66% of a physical core (and perhaps 85%-90% if the other half of the physical core emulating the logical is idle). So changing 6 cores for 6 HT can only slow you down, as a HT will never be faster than a dedicated core.

But that is unfair comparison, because with 6 HT you are only using half your capacity, while with 6 physical cores you were using all. If you want to use all, you should use all 12 HT.

Testing like this is also a bit ill defined, because you don't specify any affinity, so that you are dependent on how exactly the OS schedules threads over logical cores. Which is difficult to predict. You think nothing else is running, but usually there is, and even if it is not hogging CPU, it could lead to the OS to decide to assign two of your search threads to logical cores on the same physical core. And it is questionable ho rapidly it would decide to move it once it is running, when an other core becomes idle. So you cannot be sure that with 6 HT all your search threads will be running on different physical cores (which would be best for performance, as the other half of the physical core would be idle). It can happen that (at least during part of the time) some search threads compete each other on the same physical core. By setting affinities you can remove such uncertainties.

[hgm]

..., I suppose the statistical error is the explanation between 50 and 45.

I think it rather is because the divison by 2 is only justified if both HT on that core are working at maximum capacity. Only then a HT is 0.75 core (assuming the 50% extra raw power you quote; in my tests it was a bit less). When the other HT on that core is idle, the active HT never has to wait for any resources, and will be much faster, nearly as fast as a physical core. (Only slower because its look-ahead for out-of-order execution has been reduced, by halving the re-order buffer.)

Remember that in HT the two logical cores share almost everything, certainly all execution units, and (perhaps most detrimental) the level-1 (and 2) cache.

[Martin Thoresen]

Some math give : "HT on" give +50% raw computation power. But here, it's divide by 2 because other half is use by another engine so, 150%/2 = 75% speed : 38 sec/0.75 = 50 sec ; about the 45 sec, I suppose the statistical error is the explanation between 50 and 45.

The test states that there is no other engine running, that's the point.