Strongest MPI-capable (cluster) engine?

[Don]

Cheapest 2 machine cluster is probably buy a twin node from ebay.

That's 4 sockets in total. They're there for $200 right now. Needs CPU's.

$19 * 4 for L5420 right now on ebay.
Already comes with heatsinks and psu.
and you probably want 2 x 8 GB ram.

As it's just 2 nodes you just need an infiniband cable to connect 2 nodes.
Only above 2 nodes you need a switch.

So for $300 you have in total 40Ghz worth of core2 power cluster with 4 sockets and 2 machines and in total 16 cores.

That sounds good but I'll bet the performance is not much better than a Sandy Bridge i5. Do you have a sense of performance compared to the commodity i5's you can now get anywhere? I.E. measured by total nodes per second ratio for any chess program?

For reference, komodo 5 give me about 1.3 million nodes per second on my budget desktop box which is an Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz

For reference this is more than my i7-980x gives per core and that machine is running at 3.33 GHz.

[diep]

Cheapest 2 machine cluster is probably buy a twin node from ebay.

That's 4 sockets in total. They're there for $200 right now. Needs CPU's.

$19 * 4 for L5420 right now on ebay.
Already comes with heatsinks and psu.
and you probably want 2 x 8 GB ram.

As it's just 2 nodes you just need an infiniband cable to connect 2 nodes.
Only above 2 nodes you need a switch.

So for $300 you have in total 40Ghz worth of core2 power cluster with 4 sockets and 2 machines and in total 16 cores.

That sounds good but I'll bet the performance is not much better than a Sandy Bridge i5. Do you have a sense of performance compared to the commodity i5's you can now get anywhere? I.E. measured by total nodes per second ratio for any chess program?

For reference, komodo 5 give me about 1.3 million nodes per second on my budget desktop box which is an Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz

For reference this is more than my i7-980x gives per core and that machine is running at 3.33 GHz.

The $150 a machine is including 2 cpu's L5420. Would you buy L5420's yourself that's $19 on ebay a piece.

the older generation xeon and opteron cpu's usually are for peanuts on ebay if they do not have the latest floating point technology and are clocked not so high.

So a single sandy bridge cpu is more expensive than 4 nodes or so. Diep gets very exact benchmarked by LostCircuits. That guy is really good in benchmarking. I've not seen him make mistakes.

Realize however intel really cares about performance. Hyperthreading always works magnificent. So all i7 testresults are with hyperthreading and the cpu's even at full load they profit bigtime from hyperthreading; if you really cool cpu's well and motherboard and keep everything at around a 15C, then it eats less power and turboboost works better, whereas at home your machine inside and motherboard will be more like 50C and forget turboboost and hyperthreading as well.

In all those benchmarks, hyperthreading increases diep's NPS by at least 25% or so.

You won't get the same hyperthreading performance except when you watercool and overclock it.

Never assume that if you build the same system that without overclocking you can get the same performance like these guys - they really know how to test.

The AMD boost technology doesn't work on the other hand on all those benchmark sites.

Much of the performance in those tests therefore the RAM speed is critical, meanwhile if you have a cluster you already lose bigtime there anyway to the network, so it's not so critical. What you can't prefetch there simply is overhead you lose. An i5 doesn't offer at a cluster for chess the same benefit like it offers in a real well carried out benchmark.

In testresults from Lostcircuits you can see next:

http://www.lostcircuits.com/mambo//inde ... itstart=13

Test gets carried out everywhere with the same Diep version that SMP wise scales really well.

The fastest core5 in his test is the core i5 661 at 672k nps

The i5 is a 2 core cpu with 4 threads. And it runs default at 3.33Ghz
Its max turbo frequency is 3.6ghz so you can assume it ran at 3.6Ghz at all 4 cores.

I doubt you will manage that at home without overclocking. These guys have special motherboards to test where they can enforce turboboost always upon as they know they cool well.

Also the RAM used in the tests at lostcircuits is for all manufacturers the fastest RAM.

You will not buy such RAM.

the L5420 doesn't turboboost at all of course it's a real core2.
Now these cpu's don't scale 100% from 1 core to 4 cores. You lose something. You can see this in the test.

Just extrapolate it, that works:

At his results at lostcircuits Michael also mentions the Q6600.
Now you'll argue that's older generation cpu than the L5420.

that's entirely true, yet if you extrapolate the Q6600, realize all those
cpu's are core2's. It's the same executable you know. Core2 is a Core2.

i7 is just a core2 with on die memory controller from performance viewpoint.

Note this benchmark executable is using a few SIMD instructions but it doesn't matter much (under 1% or so).

718k nps for Q6600 is at 2.4ghz. Do that times 2 and you got my Xeon machines which are 2.5ghz times 8 cores. Somewhere around 1.4+ M nps it is.

If you scrolll up then to BEAT that nps you need a sixcore intel.
Without hyperthreading in fact a sixcore intel is on par in performance to the L5420 machine.

The newer sandy bridge cpu's have 4 memory channels.
It's not clear to me how much hyperthreading gives at those machines.

My impression is that the improved performance for Diep is improved hyperthreading performance.

Does Komodo work with hyperthreading?

If not then reduce 30% from those NPS-es you see there of the i7's.

Can you run with hyperthreading at an i5 in fact?

Now the new line of intels with 4 cores and 8 logical cores, they WILL be faster when you overclock them than the L5420 machines i got here. How much do you profit from that hyperthreading on them however as without that hyperthreading they lose 25%.

For a single node sixcore intel, i can buy 4 machines.
Those L5420 have REAL cores. That's 4 REAL cores, that of course annihilates any i5 which are 2 core cpu's.

the only cheap cpu that really is on par is the i7-2600k and newer incarnations of it, however you will need to overclock to get same performance like a real 8 core cpu.

Power consumption is similar like a L5420 machine, however you CAN over course watercool the i7-2600k very well with a relative cheap kit.

Maybe run it stable at 4Ghz in fact.

It is faster than than a L5420 * 2 cpu's machine, yet the grand total cost of such i7-2600k machine in terms of PRICE is not so interesting.

Did you consider that for a cluster too fast nodes is not interesting to have?

Many reasonable nodes is better than a few fast ones of course.

The more cores you got, the tougher it is to communicate through that single network card.

The problem of all the i7 machines is the same - the cpu cost is too high and on ebay the high clocked cpu's always keep too expensive compared to cpu's that are a lot lower clocked yet where you can put 2 of them from in a single motherboard.

My original plan to build a cluster was get some i7's relative cheap and then watercool them.

Doing the money math the real problem is the cpu costs of the i7's.

Forget anything underneath that. Realize also the i5's have far less memory channels than the i7's and they always burn up quicker if you would want to overclock.

Overclocking at a cluster - i don't need to talk about that to someone who was administrator of some big supercomputers.

[diep]

Cheapest 2 machine cluster is probably buy a twin node from ebay.

That's 4 sockets in total. They're there for $200 right now. Needs CPU's.

$19 * 4 for L5420 right now on ebay.
Already comes with heatsinks and psu.
and you probably want 2 x 8 GB ram.

As it's just 2 nodes you just need an infiniband cable to connect 2 nodes.
Only above 2 nodes you need a switch.

So for $300 you have in total 40Ghz worth of core2 power cluster with 4 sockets and 2 machines and in total 16 cores.

That sounds good but I'll bet the performance is not much better than a Sandy Bridge i5. Do you have a sense of performance compared to the commodity i5's you can now get anywhere? I.E. measured by total nodes per second ratio for any chess program?

For reference, komodo 5 give me about 1.3 million nodes per second on my budget desktop box which is an Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz

For reference this is more than my i7-980x gives per core and that machine is running at 3.33 GHz.

As for the i5-2320 it's 4 real cores running at 3.0Ghz and turboboosting to 3.3Ghz.

It has 2 memory channels, as opposed to the fast i7 sixcores sandy bridge have 4 memory channels.

It is 95 watt TDP already. That's a lot for a small chip.

So it has half the memory latency, yet of course it's 4 cores only and DDR3 is already fast.

There isn't a benchmark of them with Diep yet it is very easy to guess.

In fact it's just like a core2 for diep then, just wins some speed because DDR3 is high clocked and on die. For a cluster this is not a major argument.

Price of the cpu is the thing. If it's cheap, go for them.

3.0Ghz * 4 = 12Ghz in total.

The L5420 machines are 20Ghz and half the price of a i5-2320 system.

For clustering I still find the L5420 machines a better deal and by a large factor.

i7 is say 7% faster than core2 for Diep. That is however with 3 memory channels and this i5 has just 2.

Best compare of your i5-2320 slightly overclocked to say 3.8Ghz or so,
that's probably with an i7-2600k and then reducing for hyperthreading.

If we kick out 25% hyperthreading from benchmark result of i7-2600k of Diep that's 1.38MLN nps / 1.25 = 1.10 million nps

that's not bad Don. However there is 1 important condition. YOU HAVE TO OVERCLOCK THE I5-2320. And a lot.

So yes it's a good chip and my initial guess of 12Ghz is probably no good as you CAN easily overclock that i5-2320 i assume.

So it's then 1.1 mln / 1.4 mln = roughly 78% of the performance of my 8 core Xeon machines here.

Yet at a much much higher price.

[diep]

I checked here in shop. It's in Netherlands cheapest offer 160 euro the i5-2320.

That's too expensive.

Faster than that and the same pricerange for sure is going to be the AMD sixcores like the 1100T.

The budget edition is called X6 1045T now. It's 2.7Ghz and overclocks to a 3.xGhz easily as well.

They're 100 euro in the shops here and have 6 real cores. that's more than 4.

[syzygy]

Realize however intel really cares about performance. Hyperthreading always works magnificent. So all i7 testresults are with hyperthreading and the cpu's even at full load they profit bigtime from hyperthreading; if you really cool cpu's well and motherboard and keep everything at around a 15C, then it eats less power and turboboost works better, whereas at home your machine inside and motherboard will be more like 50C and forget turboboost and hyperthreading as well.

In all those benchmarks, hyperthreading increases diep's NPS by at least 25% or so.

You won't get the same hyperthreading performance except when you watercool and overclock it.

Are you saying that hyperthreading is more efficient at lower temperatures? If so, then that's not true.

What is true is that hyperthreading and turboboost increase power consumption and therefore produce more heat. With insufficient cooling that will lead to cpu throttling. As long as cooling is sufficient to keep temperatures at a decent level, there is no difference between 15 degrees celsius and normal temperatures.

Regarding power consumption, the less energy going into cooling the better. Cooling more simply consumes more power.

I doubt you will manage that at home without overclocking. These guys have special motherboards to test where they can enforce turboboost always upon as they know they cool well.

The BIOS of regular i7 mainboards allows you to permanently enable turboboost for all cores and to specify the multiplier. That is how i7s are overclocked. As long as there is sufficient cooling to keep temperatures within operating limits, this works just fine.

Also the RAM used in the tests at lostcircuits is for all manufacturers the fastest RAM.

You will not buy such RAM.

Why not, if one has the money... (in which case I'm not sure why bother with 4-year old L5420s).

Can you run with hyperthreading at an i5 in fact?

i5 has no hyperthreading...

Those L5420 have REAL cores. That's 4 REAL cores, that of course annihilates any i5 which are 2 core cpu's.

i5s have 4 REAL cores. I'm not sure where you are getting your processor knowledge from?

[diep]

Realize however intel really cares about performance. Hyperthreading always works magnificent. So all i7 testresults are with hyperthreading and the cpu's even at full load they profit bigtime from hyperthreading; if you really cool cpu's well and motherboard and keep everything at around a 15C, then it eats less power and turboboost works better, whereas at home your machine inside and motherboard will be more like 50C and forget turboboost and hyperthreading as well.

In all those benchmarks, hyperthreading increases diep's NPS by at least 25% or so.

You won't get the same hyperthreading performance except when you watercool and overclock it.

Are you saying that hyperthreading is more efficient at lower temperatures? If so, then that's not true.

Turboboost doesn't work at low temperatures.

Keeping everything under 20C is NOT EASY.

The cpu doesn't turboboost at all very well under full load of a chessprogram, so what these guys is enforce the turboboost using motherboards that have an option to enforce turboboost.

At those higher clock frequencies the hyperthreading gives for diep a bigger increase in nps.

For example at 3.2Ghz to 3.3Ghz it gives a 22% for diep and at 4.5Ghz that goes up to 30%+ for an i7-990x watercooled to 4.5ghz.

What is true is that hyperthreading and turboboost increase power consumption and therefore produce more heat. With insufficient cooling that will lead to cpu throttling. As long as cooling is sufficient to keep temperatures at a decent level, there is no difference between 15 degrees celsius and normal temperatures.

Normal temperature is around a 40C-50C in most cases. Especially motherboard components get affected nearby the CPU.

At 15C your system will eat around 10% less power than

the CPU's transistors work optimally at a temperature of 15C - 20C.

So either cooling it too cold or having them at normal operating temperatures, they eat more power and therefore the turboboost doesn't activate.

Now in case of these testers they ENFORCE the turboboost. Now i don't have an i7 so i cannot confirm whether normal bioses have this option.

Regarding power consumption, the less energy going into cooling the better. Cooling more simply consumes more power.

Cooling more allows more overclocking of the cpu. If you manage to watercool the cpu to 15C - 20C you can easily get it to 4.5ghz as at 15C-20C it consumes less power than when it is at 50C or above.

I doubt you will manage that at home without overclocking. These guys have special motherboards to test where they can enforce turboboost always upon as they know they cool well.

The BIOS of regular i7 mainboards allows you to permanently enable turboboost for all cores and to specify the multiplier. That is how i7s are overclocked. As long as there is sufficient cooling to keep temperatures within operating limits, this works just fine.

That's not what i mean. Wit henforce i mean that they ENFORCE the i7-2600k to run at 3.8Ghz.

In regular bios you can ALLOW it to turboboost if it sees it has enough power to do so.

Practical that means that at normal users home the turboboost does NOT function under full load and it does with them.

Also the RAM used in the tests at lostcircuits is for all manufacturers the fastest RAM.

You will not buy such RAM.

Why not, if one has the money... (in which case I'm not sure why bother with 4-year old L5420s).

First of all we discuss clusters here and then price matters a lot.

Secondly...

'buying testgear' what planet are you from?
This is testmachines.

They are under NDA until they are allowed to write about its performance.

You seem to have no clue how this testing happens. This is a 100 billion dollar business. If you misrepresent intel or AMD, they sue you to planet Mars.

So you can never have that gear until they write about it and then you still don't have it. We must make no illusions about the average computer chess guy being able to install and and get the components in a manner how they manage to test.

Forget it. You won't manage.

This is a professional multibillion dollar business.

They like to have the best available RAM. Usually when i ask them what RAM they used and i google, i can't get the RAM, or it's more expensive than the entire machine.

Can you run with hyperthreading at an i5 in fact?

i5 has no hyperthreading...

http://ark.intel.com/products/43553/Int ... -3_33-GHz)

Look 2 cores @ 4 threads.

i5-661

Those L5420 have REAL cores. That's 4 REAL cores, that of course annihilates any i5 which are 2 core cpu's.

i5s have 4 REAL cores. I'm not sure where you are getting your processor knowledge from?[/quote]

the initial i5's have 2 cores and 4 threads.
some of the sandy bridge i5's have 4 real cores.

This is all utter confusing to normal users.

Intel is tossing in huge amount of different cpu's,
yet only the i7 sixcores are fast.

The sandy bridge i5's same speed like a core2 of 4 cores at same clock for Diep. The benchmark shows that my extrapolation is very accurate, as the i5-2500k is a good comparision with the i5-2320, just it's at a different Ghz.

And it gets 1.08M nps and it boosts to 3.7Ghz.

So my extrapolation was correct.
The 8 core Xeon machines that are up for grabs for $150 on ebay, they are a lot faster than the sandy bridge i5's overclocked.

[syzygy]

Turboboost doesn't work at low temperatures.

I'm not sure this is what you meant to say.

Keeping everything under 20C is NOT EASY.

Certainly not. However, there is absolutely no need to keep everything under 20C. Keeping everything under 20C is only an enormous waste of energy.

The cpu doesn't turboboost at all very well under full load of a chessprogram, so what these guys is enforce the turboboost using motherboards that have an option to enforce turboboost.

At least with unlockable (-K) Sandybridge i7s, the BIOS normally allows you to enable turboboost for any period of time (and the multiplier you desire). But I guess what you say might be correct for other processors.

At those higher clock frequencies the hyperthreading gives for diep a bigger increase in nps.

I can believe that hyperthreading is relatively more beneficial at high frequencies where there is more opportunity for hyperthreading to hide memory latency.

For example at 3.2Ghz to 3.3Ghz it gives a 22% for diep and at 4.5Ghz that goes up to 30%+ for an i7-990x watercooled to 4.5ghz.

So that would be explained by memory being relatively slower with the cpu at 4.5Ghz as compared to the cpu at 3.3Ghz, giving a boost to hyperthreading.

The temperatures at which you reach this 4.5Ghz don't matter for speed, as long as the CPU doesn't start throttling.

Normal temperature is around a 40C-50C in most cases. Especially motherboard components get affected nearby the CPU.

As long as they stay within operating limits, so don't fail, I don't see much of a problem.

At 15C your system will eat around 10% less power than

No way! You need to add the power cost of cooling to 15C.

the CPU's transistors work optimally at a temperature of 15C - 20C.

Well, the speed of the CPU is determined by its clock rate.
I was going to add that the power consumed by a CPU is determined by the voltage going in and the number of state transitions of its transistors, and that temperature has little to do with this. However, I just learned that due to the Poole-Frenkel effect the static leakage power consumption of a CPU does increase with temperature. So indeed hotter CPUs do consume more power. Still, I would be very surprised if this increase outweighs the increased power consumption of extra cooling. Certainly it cannot outweigh the increased power consumption of cooling the CPU to 15C.

Now in case of these testers they ENFORCE the turboboost. Now i don't have an i7 so i cannot confirm whether normal bioses have this option.

I have one, and my bios does have this option.

Regarding power consumption, the less energy going into cooling the better. Cooling more simply consumes more power.

Cooling more allows more overclocking of the cpu. If you manage to watercool the cpu to 15C - 20C you can easily get it to 4.5ghz as at 15C-20C it consumes less power than when it is at 50C or above.

I bet that total power consumption is less with the CPU at 50C than with the CPU at 15C.

Also the RAM used in the tests at lostcircuits is for all manufacturers the fastest RAM.

You will not buy such RAM.

Why not, if one has the money... (in which case I'm not sure why bother with 4-year old L5420s).

'buying testgear' what planet are you from?
This is testmachines.

They are under NDA until they are allowed to write about its performance.

You seem to have no clue how this testing happens. This is a billion dollar business. If you misrepresent intel or AMD, they sue you to planet Mars.

Maybe I missed how this part of the discussion got into a subthread on cheap computing power.

[Don]

Hyperthreading is a benefit in our automated testing. We typically overprovsions 2 to 1, for example if we have 4 real cores we keep 8 matches running. It gives us more CPU throughput that running just 4. In other words we can run more 10 ply games per hour allocation 8 than 4.

Some of this could be I/O overhead however. If you allocate 4 tests on a quad and the test is hyper-fast, there may be a significant amount of time spent with communication between the engines and the tester itself. In such a case allocating more than 4 may be a benefit even if hyperthreading was not an issue. I have not done a really careful study of that to see, but it's a typical phenomenon to see idle processors when non-cpu resources are involved.

Don

Realize however intel really cares about performance. Hyperthreading always works magnificent. So all i7 testresults are with hyperthreading and the cpu's even at full load they profit bigtime from hyperthreading; if you really cool cpu's well and motherboard and keep everything at around a 15C, then it eats less power and turboboost works better, whereas at home your machine inside and motherboard will be more like 50C and forget turboboost and hyperthreading as well.

In all those benchmarks, hyperthreading increases diep's NPS by at least 25% or so.

You won't get the same hyperthreading performance except when you watercool and overclock it.

Are you saying that hyperthreading is more efficient at lower temperatures? If so, then that's not true.

Turboboost doesn't work at low temperatures.

Keeping everything under 20C is NOT EASY.

The cpu doesn't turboboost at all very well under full load of a chessprogram, so what these guys is enforce the turboboost using motherboards that have an option to enforce turboboost.

At those higher clock frequencies the hyperthreading gives for diep a bigger increase in nps.

For example at 3.2Ghz to 3.3Ghz it gives a 22% for diep and at 4.5Ghz that goes up to 30%+ for an i7-990x watercooled to 4.5ghz.

What is true is that hyperthreading and turboboost increase power consumption and therefore produce more heat. With insufficient cooling that will lead to cpu throttling. As long as cooling is sufficient to keep temperatures at a decent level, there is no difference between 15 degrees celsius and normal temperatures.

Normal temperature is around a 40C-50C in most cases. Especially motherboard components get affected nearby the CPU.

At 15C your system will eat around 10% less power than

the CPU's transistors work optimally at a temperature of 15C - 20C.

So either cooling it too cold or having them at normal operating temperatures, they eat more power and therefore the turboboost doesn't activate.

Now in case of these testers they ENFORCE the turboboost. Now i don't have an i7 so i cannot confirm whether normal bioses have this option.

Regarding power consumption, the less energy going into cooling the better. Cooling more simply consumes more power.

Cooling more allows more overclocking of the cpu. If you manage to watercool the cpu to 15C - 20C you can easily get it to 4.5ghz as at 15C-20C it consumes less power than when it is at 50C or above.

I doubt you will manage that at home without overclocking. These guys have special motherboards to test where they can enforce turboboost always upon as they know they cool well.

The BIOS of regular i7 mainboards allows you to permanently enable turboboost for all cores and to specify the multiplier. That is how i7s are overclocked. As long as there is sufficient cooling to keep temperatures within operating limits, this works just fine.

That's not what i mean. Wit henforce i mean that they ENFORCE the i7-2600k to run at 3.8Ghz.

In regular bios you can ALLOW it to turboboost if it sees it has enough power to do so.

Practical that means that at normal users home the turboboost does NOT function under full load and it does with them.

Also the RAM used in the tests at lostcircuits is for all manufacturers the fastest RAM.

You will not buy such RAM.

Why not, if one has the money... (in which case I'm not sure why bother with 4-year old L5420s).

First of all we discuss clusters here and then price matters a lot.

Secondly...

'buying testgear' what planet are you from?
This is testmachines.

They are under NDA until they are allowed to write about its performance.

You seem to have no clue how this testing happens. This is a 100 billion dollar business. If you misrepresent intel or AMD, they sue you to planet Mars.

So you can never have that gear until they write about it and then you still don't have it. We must make no illusions about the average computer chess guy being able to install and and get the components in a manner how they manage to test.

Forget it. You won't manage.

This is a professional multibillion dollar business.

They like to have the best available RAM. Usually when i ask them what RAM they used and i google, i can't get the RAM, or it's more expensive than the entire machine.

Can you run with hyperthreading at an i5 in fact?

i5 has no hyperthreading...

http://ark.intel.com/products/43553/Int ... -3_33-GHz)

Look 2 cores @ 4 threads.

i5-661

Those L5420 have REAL cores. That's 4 REAL cores, that of course annihilates any i5 which are 2 core cpu's.

i5s have 4 REAL cores. I'm not sure where you are getting your processor knowledge from?

the initial i5's have 2 cores and 4 threads.
some of the sandy bridge i5's have 4 real cores.

This is all utter confusing to normal users.

Intel is tossing in huge amount of different cpu's,
yet only the i7 sixcores are fast.

The sandy bridge i5's same speed like a core2 of 4 cores at same clock for Diep. The benchmark shows that my extrapolation is very accurate, as the i5-2500k is a good comparision with the i5-2320, just it's at a different Ghz.

And it gets 1.08M nps and it boosts to 3.7Ghz.

So my extrapolation was correct.
The 8 core Xeon machines that are up for grabs for $150 on ebay, they are a lot faster than the sandy bridge i5's overclocked.[/quote]

[mike_bike_kite]

Say 16 real cores @ 4.5Ghz or so, that's 8 * 9 = 72Ghz.

but will it play Crysis?
Mike

[diep]

Say 16 real cores @ 4.5Ghz or so, that's 8 * 9 = 72Ghz.

but will it play Crysis?
Mike

It plays Toaster very well.