Interest in using GPU's for EGTB generator

[vb4]

I have been in communication with a big company who appears to be willing to put up some big hardware for the purpose of being a sponsor to generate the 7 piece EGTB utilizing GPU's if someone here is willing or knows someone that could write the application. I dont know much about GPU's but this looks like an opportunity. These people are hardware experts and know very little about software and what it entails. The people I have spoken to have shown a genuine interest in getting involved and have also told me that the hardware is already in place. If some of you can provide feedback as to how we might want to approach this and if it is feasible or not I wiould appreciate it.

Looking forward to your responses.

Les

[hgm]

Why would you want to use GPUs to generate tablebases? EGTB building is a storage / I/O problem, not a computational problem. Even if you do it in RAM with a single CPU, the CPU is twiddling its thumbs 90% of the time waiting for a memory access...

[vb4]

Hi HG,

Like I said I know nothing about GPU's and only thought that I post here to see if their is any way for us to take advantage of tis opportunity. My understanding is that GPU's outperform CPU's by a considerable margin and thought I post this and get feedbacks from some of the experts out here.

Les

[wgarvin]

Hi HG,

Like I said I know nothing about GPU's and only thought that I post here to see if their is any way for us to take advantage of tis opportunity. My understanding is that GPU's outperform CPU's by a considerable margin and thought I post this and get feedbacks from some of the experts out here.

Les

Only for certain kinds of highly parallelizable tasks. For general-purpose tasks, CPUs are better.

By the way, a plug for hgm's Leapfrog algorithm for a 7-man generator: http://home.hccnet.nl/h.g.muller/EGT7/7-men.html

If you read those pages, you'll get an idea of just how complicated it would be to generate 7-man EGTBs with the hardware we have today. The computation has to be carefully arranged so that it can be done with the limited amounts of RAM we have today, and also to try and keep wasted I/O to a minimum.

Its similar to the complexity that the Chinook team faced when they generated their endgame databases for checkers--it took them many months using dozens (perhaps hundreds) of idle computers at the university. Machines with hundreds of megabytes of RAM were rare back then, so all of the databases had to be chopped into 'slices' that could be computed separately. Nowadays you could do away with alot of that complexity and just use a 64-bit machine with anywhere from 4 to 12 GB of memory. Probably any modern desktop machine could generate all of those databases in a week or something. What was an impressive accomplishment with the less powerful computers they had back then, would be almost mundane to do today.

The situation is similar with chess endgames; for 4- and 5-piece, you can generate them reasonably quickly nowadays. For 6-piece it still takes a long time, but relatively simple generators still suffice. Generating some 7-piece ones might be barely feasible now, but only if you are clever about it (thus, Leapfrog).

[Edmund]

Why would you want to use GPUs to generate tablebases? EGTB building is a storage / I/O problem, not a computational problem. Even if you do it in RAM with a single CPU, the CPU is twiddling its thumbs 90% of the time waiting for a memory access...

How about keeping the processors busy by searching more than just one ply per iteration?

Though I don't think this would work with GPUs due to the necessary large shared memory.

[vb4]

Hi Wylie,

Thanks for your explanation. Maybe someone else out here may have a differnet take on it or some clever idea.

[vb4]

Hi Ed,

The company interested in soponsoring a prject like this has told me they have several Linux clusters with 500+ processing cores. Cores range from 2.33-3.0Ghz and have 1-2GB of RAM per core. They also mentioned over the phone, they recently setup 10 nodes with GPUs. Each node as one nVidia GTX285 GPU with 240 stream processors and 2GB of RAM. Whether this info helps any of you out here you can see they appear to be very interested.

[Dann Corbit]

Have you made the same post at the EGTB forum?
I guess you might also get interest there.

[Gian-Carlo Pascutto]

Hi Ed,

The company interested in soponsoring a prject like this has told me they have several Linux clusters with 500+ processing cores. Cores range from 2.33-3.0Ghz and have 1-2GB of RAM per core. They also mentioned over the phone, they recently setup 10 nodes with GPUs. Each node as one nVidia GTX285 GPU with 240 stream processors and 2GB of RAM. Whether this info helps any of you out here you can see they appear to be very interested.

Hmm, nice machine to run Sjeng on

[Edmund]

Hi Ed,

The company interested in soponsoring a prject like this has told me they have several Linux clusters with 500+ processing cores. Cores range from 2.33-3.0Ghz and have 1-2GB of RAM per core. They also mentioned over the phone, they recently setup 10 nodes with GPUs. Each node as one nVidia GTX285 GPU with 240 stream processors and 2GB of RAM. Whether this info helps any of you out here you can see they appear to be very interested.

For the 500+ cpus it would probably be best if they worked on different independent endgames in parallel. This way you sould be getting the best scaling.

The GPUs are more challenging, because they have to work on the same endgame. What do the others think of my proposal to use the additional processors to do more than just one leap per itereration? What benefit could we expect?