You cannot do time limited testing when using 1 core @ hyperthreading. If 1 proces by accident gets a real core without another proces using the other logical core very well, then i don't know about your IPC, but suddenly 1 proces then gets all system time.Don wrote:Hyperthreading is a benefit in our automated testing.
So say you have a system with 4 real cores @ 8 logical cores.
If you run your engine at 1 core, without another engine on it, IPC in case of Diep is around a 1.5 (i use rounded off numbers now to illustrate the example, actually Diep's IPC is far above 1.5 at core2 cpu's and at i7 it's far above 1.7 with 2 logical cores using 1 core).
When 2 processes run at 1 core, so both getting a logical core, their *total* IPC is 1.7. So the IPC for a single proces is 0.85.
If you run matches now just using logical cores, regurarly some cores will get that IPC=1.5 at 1 or more moves and their opponent will be 0.85 at that moment. So that isn't a fair match to say polite.
It's not factor 2 advantage, yet nearly, for most likely some critical moves.
This creates huge noise in your testresults and it happens nonstop.
To filter that out would require a million games or so if not more, this for a single run.
Hyperthreading creates huge non-determinism; that creates huge noise.
Anything that works realtime which requires accurate measurement won't work well with hyperthreading as measuring gets more complicated, because your proces X never knows whether another proces runs AT THIS MOMENT at the other logical core or not. It switches all that quick that you just never know about all this.
Note that the i5 @ 2300 series doesn't have hyperthreading. Neither does the Xeon 5400 series.
That's a huge advantage for speedy testing.
If you run single core matches, just go for the system with most combined Ghz i'd argue.
That's what i did do and i don't regret it.
I'm a bit amazed that L5420 still is best deal on ebay there. The cpu's are 19 dollar a piece and it's 4 x 2.5Ghz and a single machine can have 2 of them and it's a low power CPU so power usage isn't high.
That's 8 * 2.5Ghz = 20Ghz for $38 in cpu costs.
p.s. of course fastest way to test for you would involve quite some programming work; you could port your engine to CUDA or something and then test at thousands of cores at same time. It's a tad of work, but it'll speedup your testing factor 100 or so.
