Page 1 of 2

Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 4:33 am
by Dann Corbit

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 5:28 am
by dragontamer5788
If anyone is a build-your-own PC person, the motherboard is a Gigabyte TRX40 Designare.

Two GPUs will be x16, while two will be x8.

Image

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 9:22 am
by smatovic
These Bizons play in the same league as the 4-way Nvidia DGX Station and the
upcoming Apple Mac Pro, but to a fraction of the price...

--
Srdja

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 9:00 pm
by Dann Corbit
dragontamer5788 wrote: Tue Nov 26, 2019 5:28 am
If anyone is a build-your-own PC person, the motherboard is a Gigabyte TRX40 Designare.

Two GPUs will be x16, while two will be x8.

Image
Thanks for looking into that.
While the current GPUs can be handled at full throttle over the x8 channel, future GPUs maybe not.
So I will wait for the WRX boards instead.

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 9:38 pm
by dragontamer5788
Dann Corbit wrote: Tue Nov 26, 2019 9:00 pm
dragontamer5788 wrote: Tue Nov 26, 2019 5:28 am Two GPUs will be x16, while two will be x8.
Thanks for looking into that.
While the current GPUs can be handled at full throttle over the x8 channel, future GPUs maybe not.
So I will wait for the WRX boards instead.
I've been making progress with my AB-parallel algorithm, and I don't think it is communication bound. AB-search is "tree-like" search, and the amount of communication between threads seems to be small, even if I have CPU + GPU cores working together.

With regards to the premier GPU code: LeelaZero is a CNN, which is an extremely compute-heavy algorithm that doesn't seem to need much bandwidth (if fully optimized). LeelaZero is ultimately only pushing 50knps (or less) per GPU, while a PCIe 3.0 x8 slot has a theoretical bandwidth of ~7GBps... maybe 5GBps-per-direction in practice.

So assuming one-way communications (worst-case scenario, since PCIe is duplex and can perform parallel transfers in both directions), that is 5GBps of PCIe 3.0 x8 communication delivering 50,000 nodes per second, or roughly 100MB per node. Yeah... I think you're fine. I don't expect chess to be PCIe-bottlenecked at all.

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Tue Nov 26, 2019 9:57 pm
by OneTrickPony
Any way to get 256GB of RAM on those? I am clueless about hardware and when looking for dimms which are 32GB (so I could get 8x32) I only see some "dedicated" dies running at 2400Mhz (which seems kinda slow?).
Did anyone made a TR build with 256GB of RAM already?

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Wed Nov 27, 2019 4:26 am
by MikeB
their price is high ... e.g., the upgrade to the 3970 from 3960 should be $600 to $ 800 max, not $1200...

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Wed Nov 27, 2019 5:01 am
by Zenmastur
OneTrickPony wrote: Tue Nov 26, 2019 9:57 pm Any way to get 256GB of RAM on those? I am clueless about hardware and when looking for dimms which are 32GB (so I could get 8x32) I only see some "dedicated" dies running at 2400Mhz (which seems kinda slow?).
Did anyone made a TR build with 256GB of RAM already?
CORSAIR Vengeance LPX 256GB (8 x 32GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model CMK256GX4M8E3200C16 for $1235 on newegg.

https://www.newegg.com/corsair-256gb-28 ... 6820236595

Regards,

Zenmastur

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Wed Nov 27, 2019 5:28 am
by jdart
MikeB wrote: Wed Nov 27, 2019 4:26 am their price is high ... e.g., the upgrade to the 3970 from 3960 should be $600 to $ 800 max, not $1200...
Yeah, it is very high. I am thinking about building a 32-core Threadripper system, but I'd probably use to replace my 2013-era Opteron system and reuse its big server case and some of its components, so a CPU based system wouldn't cost over $3k I would think. Adding GPU would bump it up but it still wouldn't hit their $5600 base price.

--Jon

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Posted: Wed Nov 27, 2019 9:22 am
by OneTrickPony
CORSAIR Vengeance LPX 256GB (8 x 32GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model CMK256GX4M8E3200C16 for $1235 on newegg.
This is great as that's the only one I've seen which is clocked at 3200Mhz.
Sadly it seems not available in Europe yet but I hope it soon will be.