Fantacising about a testing cluster!

[Michael Sherwin]

I want to be able to run a two month test for the most promising beta that is produced each week. So ten cpus should be enough. They should be as cheap as possible (celerons maybe) with no more than 256 MB each. I imagine that they all could connect to a main computer through a simple network that is not one of the ten. In my fantasy they do not have keyboards or monitors and only have cheap small capasity harddrives. All ten are controlled from the main computer and any beta can be assigned very easily to any of the ten cpus for testing. And it is a MS Windows system (really).

Is this just pure fantacy or can it be done? And for how much?

[LarsA]

Dell has som interesting server offers for small businesses wich they also will sell to private persons.

The very basic model starts at 450 usd. This has a celeron processor and 1024 MB ram.

For just 600 usd you can get them with dual core processors (Pntium D), and at 750 usd with Xeon 3000-series processors (server edition of core 2 duo)

[sje]

I predict you'll have a big bill for air conditioning this summer.

--------

Mac OS/X has a free included distributed computing framework called Xgrid and a number have people have used this to write distributed applications designed to run on a LAN. I don't have much personal experience with this myself, though.

For relatively cheap clustering, some web host providers are using arrays of Mac Mini boxes (1.67/1.83 GHz CoreDuo, starting at US$600). Each machine uses only about 80 watts at full load and can run Windows if needed. However, it might be better to wait for the Core2Duo model and so have the option for 64 bit processing.

I think the Mac Mini is a fair value for the price, but like other relatively cheap boxes and unlike all other Macs, it uses vampire video. That's not a big issue for cluster use.

--------

The older Pentium implementations are cheap, but they're cheap for a reason: a definite lack of performance compared to newer Core2Duo and Xeon designs. I have one of the older Pentium designs, a US$400 Compaq Presario 3 GHz Pentium 4; it even came with a keyboard and mouse. It runs decently with Ubuntu 6.10, but I don't think it would be worth it to buy ten of them.

Perhaps you might consider a blade server. The extra cost can be justified for the greater reliability, lower total power usage, and space savings.

[bob]

I want to be able to run a two month test for the most promising beta that is produced each week. So ten cpus should be enough. They should be as cheap as possible (celerons maybe) with no more than 256 MB each. I imagine that they all could connect to a main computer through a simple network that is not one of the ten. In my fantasy they do not have keyboards or monitors and only have cheap small capasity harddrives. All ten are controlled from the main computer and any beta can be assigned very easily to any of the ten cpus for testing. And it is a MS Windows system (really).

Is this just pure fantacy or can it be done? And for how much?

It can obviously be done, as we have a cluster of 128 nodes, each node with 4 gigs of ram and dual xeon processors.

However, 10 is going to take quite a while to run enough games to conclude version X+1 is better than version X. Takes a large number of games to conclude that...

[Michael Sherwin]

Well I do want to run one beta on one processor for two months! The compromise that I had in mind was only running blitz games. One month of 40/4 and one month of 4'4" which should give enough games. With out a 128 nodes to work with some compromise must be made in order to get enough games!

[Michael Sherwin]

Summer may be a problem! Since they all would be in a spare room with two windows I was just going to keep the door shut and use lots of window fans. In winter time the cost to run them should be negligible!

Lots of testers use celerons to test with and it seems that their results are just as valid. Tony Thomas test on a celeron and also uses almost exclusively 1'1" time limits and his results are very close to the truth.

There are engines that do not do well at blitz on slow machines, however, since I am only testing for a relative difference between beta versions and not for accuracy of rating there should not be a problem.

And bare minimum nodes as I originally described them should even be less expensive than what has been suggested so far. The cheaper the better. So, a simple celeron (sempro?) box for US $250 would be fantastic!

After having said all that, it would be a shame to not run 64 bit test! But once again everything is relative and if the 32 bit version does better then the 64 bit version should also be better.

[sje]

You might try connecting with a nearby school or university. They often upgrade and will auction off older machines, many or all of the same model at the same time.

[Michael Sherwin]

You might try connecting with a nearby school or university. They often upgrade and will auction off older machines, many or all of the same model at the same time.

Good idea!

Thanks!

[Denis P. Mendoza]

Michael,

I've also dreamed about it, like the one in this old site (using Linux):
http://lcdx00.wm.lc.ehu.es/~svet/beowulf/

But now that Microsoft started its own clusters scheme, it will be just be a DIY project for you or anyone interested. These 2 links would be a good start.

http://www.csm.ornl.gov/pvm/
http://www-unix.mcs.anl.gov/mpi/mpich/

I hope I'll build my own someday.

Denis

[bob]

I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.