Configure a threadripper 3 workstation with up to 4 GPUs

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Dann Corbit
Posts: 10242
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Configure a threadripper 3 workstation with up to 4 GPUs

Post by Dann Corbit » Tue Nov 26, 2019 3:33 am

Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

dragontamer5788
Posts: 201
Joined: Thu Jun 06, 2019 6:05 pm
Full name: Percival Tiglao

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by dragontamer5788 » Tue Nov 26, 2019 4:28 am

If anyone is a build-your-own PC person, the motherboard is a Gigabyte TRX40 Designare.

Two GPUs will be x16, while two will be x8.

Image

smatovic
Posts: 1031
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by smatovic » Tue Nov 26, 2019 8:22 am

These Bizons play in the same league as the 4-way Nvidia DGX Station and the
upcoming Apple Mac Pro, but to a fraction of the price...

--
Srdja

Dann Corbit
Posts: 10242
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by Dann Corbit » Tue Nov 26, 2019 8:00 pm

dragontamer5788 wrote:
Tue Nov 26, 2019 4:28 am
If anyone is a build-your-own PC person, the motherboard is a Gigabyte TRX40 Designare.

Two GPUs will be x16, while two will be x8.

Image
Thanks for looking into that.
While the current GPUs can be handled at full throttle over the x8 channel, future GPUs maybe not.
So I will wait for the WRX boards instead.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

dragontamer5788
Posts: 201
Joined: Thu Jun 06, 2019 6:05 pm
Full name: Percival Tiglao

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by dragontamer5788 » Tue Nov 26, 2019 8:38 pm

Dann Corbit wrote:
Tue Nov 26, 2019 8:00 pm
dragontamer5788 wrote:
Tue Nov 26, 2019 4:28 am
Two GPUs will be x16, while two will be x8.
Thanks for looking into that.
While the current GPUs can be handled at full throttle over the x8 channel, future GPUs maybe not.
So I will wait for the WRX boards instead.
I've been making progress with my AB-parallel algorithm, and I don't think it is communication bound. AB-search is "tree-like" search, and the amount of communication between threads seems to be small, even if I have CPU + GPU cores working together.

With regards to the premier GPU code: LeelaZero is a CNN, which is an extremely compute-heavy algorithm that doesn't seem to need much bandwidth (if fully optimized). LeelaZero is ultimately only pushing 50knps (or less) per GPU, while a PCIe 3.0 x8 slot has a theoretical bandwidth of ~7GBps... maybe 5GBps-per-direction in practice.

So assuming one-way communications (worst-case scenario, since PCIe is duplex and can perform parallel transfers in both directions), that is 5GBps of PCIe 3.0 x8 communication delivering 50,000 nodes per second, or roughly 100MB per node. Yeah... I think you're fine. I don't expect chess to be PCIe-bottlenecked at all.

OneTrickPony
Posts: 116
Joined: Mon Apr 29, 2013 11:29 pm

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by OneTrickPony » Tue Nov 26, 2019 8:57 pm

Any way to get 256GB of RAM on those? I am clueless about hardware and when looking for dimms which are 32GB (so I could get 8x32) I only see some "dedicated" dies running at 2400Mhz (which seems kinda slow?).
Did anyone made a TR build with 256GB of RAM already?

MikeB
Posts: 3654
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by MikeB » Wed Nov 27, 2019 3:26 am

their price is high ... e.g., the upgrade to the 3970 from 3960 should be $600 to $ 800 max, not $1200...

Zenmastur
Posts: 530
Joined: Sat May 31, 2014 6:28 am

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by Zenmastur » Wed Nov 27, 2019 4:01 am

OneTrickPony wrote:
Tue Nov 26, 2019 8:57 pm
Any way to get 256GB of RAM on those? I am clueless about hardware and when looking for dimms which are 32GB (so I could get 8x32) I only see some "dedicated" dies running at 2400Mhz (which seems kinda slow?).
Did anyone made a TR build with 256GB of RAM already?
CORSAIR Vengeance LPX 256GB (8 x 32GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model CMK256GX4M8E3200C16 for $1235 on newegg.

https://www.newegg.com/corsair-256gb-28 ... 6820236595

Regards,

Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

jdart
Posts: 3864
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by jdart » Wed Nov 27, 2019 4:28 am

MikeB wrote:
Wed Nov 27, 2019 3:26 am
their price is high ... e.g., the upgrade to the 3970 from 3960 should be $600 to $ 800 max, not $1200...
Yeah, it is very high. I am thinking about building a 32-core Threadripper system, but I'd probably use to replace my 2013-era Opteron system and reuse its big server case and some of its components, so a CPU based system wouldn't cost over $3k I would think. Adding GPU would bump it up but it still wouldn't hit their $5600 base price.

--Jon

OneTrickPony
Posts: 116
Joined: Mon Apr 29, 2013 11:29 pm

Re: Configure a threadripper 3 workstation with up to 4 GPUs

Post by OneTrickPony » Wed Nov 27, 2019 8:22 am

CORSAIR Vengeance LPX 256GB (8 x 32GB) 288-Pin DDR4 SDRAM DDR4 3200 (PC4 25600) Desktop Memory Model CMK256GX4M8E3200C16 for $1235 on newegg.
This is great as that's the only one I've seen which is clocked at 3200Mhz.
Sadly it seems not available in Europe yet but I hope it soon will be.

Post Reply