Fantacising about a testing cluster!

[krazyken]

You can get cheap windows 2000 boxes, PIII 1Ghz, for $80.

I don't remember any PIII boxes with 256 MB ram each. Can they hold that much?

Sure they can hold that much they usually use PC133 RAM and can usually hold 512-1024MB

But, I guess that you are just making a point about price! Also older opreating systems (and maybe newer ones too) would not work with out a keyboard or monitor.

Not entirely true. You could set up the boxes with a KVM so you can switch one Keyboard/Monitor/Mouse between all the boxes. Probably will cost about $250 extra. If you have USB Keyboard and mouse (PS2 will work as well, but not officially), you just need to have the video plugged in at boot time, and can hot swap the cables if needed. Tweaking settings in BIOS may be needed. Also possible you can install VNC on all the machines and control them via network.

I could buy all the pieces that I think would be required, however, I doubt that I would have a working system when it was put together. How would I find out how to make it all work and exactly what I would need to buy. A book maybe? I doubt if Dell and Gateway would work with or care about antiquated equipment that they no longer deal with.

If you are not versed in computer hardware and are not comfortable in doing things like popping it open to install RAM, your best avenue of support is the "buy the local hardware guru a pizza" trick. Although there are oodles of books for upgrading and maintaining hardware. Just look around for used computers, there are bunches of them out there, which frequently need nothing more than a RAM upgrade to get them to do what you want.

[Tony]

Some older systems need a keyboard to boot, but they may not need a keyboard to run once booted. ....

Yeah, I remember that one. "Keyboard not present. Press F1 to continue"



Tony

[bob]

I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.

Hi,

At a rate 6-8 games per day you'll get to 210 games in 30 days, so 30 days should be enough to determine wich oponent is better.

If the two versions are close, 200 games is nowhere near enough to say which version is better. I'm working on something for the JICGA right now that shows some interesting results based on this very topic. I can produce some 200+ game matches where version A wins one, version B wins the other. Both versions playing 80 games against each of 3 opponents, using 40 starting positions, each played once with each color. 240 games total. Not always enough. In fact, rarely ever enough unless something is broken to make one version clearly worse.

Hi,

From your previous post I understood that you tough something between 100 and 200 games was enough. I probably misunderstood what you meant.

Out of curiosity, how many games (and wich kind of results) do you think are needed to assert that version N+1 is an improvment over version N?

Give me a while to find that answer. I am working on it right now, in fact. What I am doing is playing 2-game matches using 40 starting positions for 80 games per match. I am playing 80 such matches for each opponent. Then 40 matches but with 4 games per opponent per position per match. Then 20 matches with 8 games per match, etc.

My intent is to determine how many games it takes to produce a stable result. I can certainly tell you that 80 games and 160 games are nowhere near enough. But an exact answer will take a while.

80 matchs X 2 games X 40 positions X 3 opponents = 19, 200 games.

Fortunately I can play 256 at a time which takes about as long as it would to play 75 games one at a time, which is not too bad. Obviously my hope is that I won't need 19200 games to decide "good" or "bad". But I absolutely need more than 80-160. So I am interested in finding a statistically valid minimum number of games needed...

More once I have got the data...

[abb]

You might try connecting with a nearby school or university. They often upgrade and will auction off older machines, many or all of the same model at the same time.

A surprisingly common find on campus is the wondrous "Forgotten (or Underutilized) Computer Lab." You can find them tucked away all over the place: quietly humming, sucking up electricity and distributing viruses/spam/worms...but not much else. ;)

When I was an undergrad, I worked w/ tech support in one of our larger departments/schools, and there were at least 3 of these orphaned labs (8 systems each) +/- 2 flights of stairs from my office. Students and profs all had laptops, flying cars, etc, so they'd stopped using the labs years ago. But the annual line-item in the budget kept them maintained (and kept me employed!) ad infinitum; who was I to complain? :)

In grad school, my assistantship (once again) put me next-door to an unused lab, which I was responsible for maintaining. After a few weeks of watching 20+ (fairly high-end) systems run nothing but NOP and Windows Update, I got permission to "play around" with Cluster Knoppix (and other LiveCD/DVD tools), "just so long as it didn't interfere with the [nonexistent] students needing to use the lab."

I bring this up for a couple of reasons:

First, from my experience, this whole "orphaned lab" syndrome seems pretty widespread... Now, you probably don't want to build your long-term Cluster Empire upon borrowed/temporary access like this, but it might offer a nice cheap/free environment for you to 'experiment' with one technology/software/architecture or another, so you can see what's available, what's feasible, etc, prior to shelling out the cash...

Second, although I know you said you're married to using Windows, it *really* might be worth it to check out this Cluster Knoppix (and/or various other linux-based clustering solutions) stuff. I say this based solely on the following:

• In under an hour, I went from a heterogeneous network of 20 locked-down Win2K workstations doing nothing to a fully functional/automatically configured, 20-node cluster, eager to do my evil, evil bidding.
• 45 minutes of said hour was spent downloading the ISOs and burning the LiveCDs needed to boot everything.
• Not counting the cost of said CD-Rs, the entire process was free.
• The initial setup/boot/configure process required almost no knowledge of unix/linux or weird-cluster-setup-issues.
• The year was 2001.

That last point really cinches it for me -- I haven't played with clustering technologies much since grad school, but for god's sake: if a linux newbie with no experience in mosix/mfs/dfsa could do all that (above), for free, 6+ years ago...I can only assume that the tools/options available today are at least worth checking out, if only as a free sunday-afternoon experiment.

Worst case scenario, you wasted an hour or two.... Best case scenario, you get added visibility into the clustering options available to you, discover a way to use commodity/cheaper hardware and/or realize other opportunities for cost savings that you might not have considered otherwise -- enabling you to make even wiser purchase decisions later, should you decide to build/fund such a thing...


And you could always use a Windows box to 'control' the cluster via ssh/vnc whatever... If you can find a cheap way to borg up 20 computers and still have the pretty/easier windows interface you're used to, well...who really cares what's "under the hood"? You click the icon that says "tell the black box to play chess for a month" and the black box plays chess for a month... Pretty icon + desired result = enough for me.

• • Besides, a famous prof once told me: "deep down, [computers] are all powered by black magic and voodoo, anyway"-- and who am I to argue? :) ...Favorite CS quote ever, btw.

I'd love to play with this stuff again... I mean, sure, folks are right when they suggest you won't be overpowering 60billion processor blade servers with commodity hardware and ethernet anytime soon (without burning your house to the ground in the process, thanks to the from the heat and electricity requirements of all those boxes, lol) -- but it would be a fun experiment nonetheless.

Find a way to let others connect remotely and you might be able to get nerds like me (who lack the space and electricity to commit to such a project, and yet, are still nerdy enough to be intrigued by it) -- you could probably find a few who'd play around (aka "help") if you wanted to go at it the F/OSS route.

• • "An open source approach to clustered software development on near-free hardware, for purposes of analyzing/evaluating/grading the effectiveness of chess software, in the absolute *nerdiest* way possible." Lol, I can see the SourceForge page already. :)

cheers and good luck, whatever route you decide to take!

[Marc Lacrosse]

I want to be able to run a two month test for the most promising beta that is produced each week. So ten cpus should be enough. They should be as cheap as possible (celerons maybe) with no more than 256 MB each. I imagine that they all could connect to a main computer through a simple network that is not one of the ten. In my fantasy they do not have keyboards or monitors and only have cheap small capasity harddrives. All ten are controlled from the main computer and any beta can be assigned very easily to any of the ten cpus for testing. And it is a MS Windows system (really).

Is this just pure fantacy or can it be done? And for how much?

You should have a look at http://www.cs.wisc.edu/condor/.

Then try to convince a large site with many computers to run such a system for you.

There are more than 1000 computers doing almost nothing at night in my hospital.

I often dreamed having them performing silent chess analyses for me...

But I did not dare so far to ask the computing team

... Sure they would reject it for security and cost reasons

Marc