44 elo swing depending on hardware!

[Modern Times]

It would be very disappointing if Houdini 4 plays with almost equal strength on all computers, like H 3 does at present !

That statement is false. Maybe Houdini doesn't improve as much as other engines, but to say there is no improvement at all is not true.

[shrapnel]

It would be very disappointing if Houdini 4 plays with almost equal strength on all computers, like H 3 does at present !

That statement is false. Maybe Houdini doesn't improve as much as other engines, but to say there is no improvement at all is not true.

Hi Ray
Read my statement again, carefully. I used the word 'almost', see.
Of course there is improvement in Houdini strength with increasing strength of Hardware, but not at the same rate as with Komodo 6 !

[Modern Times]

Hi Ray
Read my statement again, carefully. I used the word 'almost', see.
Of course there is improvement in Houdini strength with increasing strength of Hardware, but not at the same rate as with Komodo 6 !

OK yes - indeed my gut feel based on results I've seen from others is that Komodo 6 improves more, but I have no proof of that. Based on that gut feel, I think Komodo 6 will win TCEC.

Houdini 3 certainly improves as you would expect up to 6 cores, but beyond that I don't know. Once you get into longer time controls and > 8 cores, the Elo gap between engines can close up, and the draw rate can increase, so it gets even harder to tell.

[Vinvin]

meaning one test per thread rather than per core

Do you mean you use hyper-threading ?

Yes.

Using HT for engine match (even 1 thread) is certainly very bad for serious testing : an engine can get more CPU if the other engine running on the same CPU is more idle (between 2 games, when accessing EGTB, ... ). Some side effects has been nicely describe here : http://www.talkchess.com/forum/viewtopi ... che#538377

[pohl4711]

I may have to try that, although I have no experience with turning off Hyperthreading. Does turning it off increase, decrease, or leave roughly unchanged the nodes per second (assuming you run the same number of threads as cores in each case)?
How do the testing groups handle this? Do they leave Ht on or turn it off?

Engines running in mp-mode can gain a little bit nodes per second with HT on, but engines running in singlecore-mode (with some games/engines running in parallel (like my LS-ratinglist-tests with the LittleBlitzerGUI)) gain around 10% more nodes with HT off (!), because each singlecore-engine-task runs on one CPU-core and is not splitted by Windows (splitting is work for the system and so it is overhead!). So if you switch HT off, you will get a little bit more engine-speed and a clear allocation of each engine to a specific CPU-core (which makes it possible to use one CPU for one LBG-instance...).

So Hyperthreading may be good for one engine running on the complete system (on all cores), but for testwork with singlecore-engines, it is really, really bad!

Best - Stefan

[Michel]

Using HT for engine match (even 1 thread) is certainly very bad for serious testing :

This is not so clear. On fishtest there are people that use hyperthreading, apparently without detrimental effects, as shown by the statistical tests.

[Vinvin]

Using HT for engine match (even 1 thread) is certainly very bad for serious testing :

This is not so clear. On fishtest there are people that use hyperthreading, apparently without detrimental effects, as shown by the statistical tests.

It's clear that HT add noise to the results. Adding noise in a match mean that the results will be closer to 50% (because of more random moves : sometimes weaker, sometimes stronger)

[Modern Times]

Using HT for engine match (even 1 thread) is certainly very bad for serious testing :

This is not so clear. On fishtest there are people that use hyperthreading, apparently without detrimental effects, as shown by the statistical tests.

It's clear that HT add noise to the results. Adding noise in a match mean that the results will be closer to 50% (because of more random moves : sometimes weaker, sometimes stronger)

No serious tester that I know of uses HT.

When Larry tries to rationalise why the ratings lists differ from his results, the implication being that his results are more "correct" - well now I know that is in doubt.

[Michel]

No serious tester that I know of uses HT.

Trying to be on the safe side is one thing. Claiming for a fact that HT hurts without presenting any supporting evidence is something quite different. Note that I presented some actual verifiable evidence for the _opposite_ viewpoint.

[Modern Times]

No serious tester that I know of uses HT.

Trying to be on the safe side is one thing. Claiming for a fact that HT hurts without presenting any supporting evidence is something quite different. Note that I presented some actual verifiable evidence for the _opposite_ viewpoint.

"No serious tester that I know of uses HT" - yes, this is simply about being cautious.