GPU rumors 2021

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, chrisw, Rebel

Werewolf
Posts: 1936
Joined: Thu Sep 18, 2008 10:24 pm

Re: GPU rumors 2021

Post by Werewolf »

Hai wrote: Mon Dec 02, 2024 10:04 pm
Werewolf wrote: Mon Dec 02, 2024 3:32 pm Well, having just browsed the Lc0 Discord I am sad to see BT5 has been cancelled for the time being. This makes the latest 5090 much less interesting now :cry:
This makes the MacBook Pro 16-inch M4 MAX and the Mac Studio M4 ULTRA much more interesting.
1) No it doesn't. Why would it?
2) Rumours online is that the M4 Ultra has been cancelled in typical Apple fashion putting profit above power users.
smatovic
Posts: 3030
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPU rumors 2021

Post by smatovic »

One on supercomputers and FLOPS*:

gigaFLOPS
The first gigaFLOPS was achieved by the NEC SX2 and Cray 2 supercomputers from 1985, both via vector-processors, the former with 1 gigaFLOPS and 512 MB RAM the latter with 2 gigaFLOPS and 1 GB RAM.

https://en.wikipedia.org/wiki/NEC_SX
https://en.wikipedia.org/wiki/Cray-2

teraFLOPS
The first teraFLOPS was achieved by the ASCI Red supercomputer from 1997 by Intel @ Sandia National Laboratories, ~10,000 Pentium Pro CPUs@200MHz, here we see the switch from single supercomputers to clusters of nodes.

https://en.wikipedia.org/wiki/ASCI_Red

petaFLOPS
The first petaFLOPS was achieved by Roadrunner from 2008 by IBM @ Los Alamos National Laboratory, with 12,960 IBM PowerXCell 8i accelerator boards, here we see the switch to heterogeneous CPU+accelerator architecture.

https://en.wikipedia.org/wiki/Roadrunne ... rcomputer)

exaFLOPS
The first (official) exaFLOPS was achieved by Frontier from 2022 by Cray+AMD @ Oak Ridge National Laboratory , with 9,472 AMD Epyc CPUs plus 37,888 AMD Instinct MI250X GPUs.

https://en.wikipedia.org/wiki/Frontier_(supercomputer)

Outlook first zettaFLOPS?
Idk :) but it looks atm like this will be a machine for training AIs, either a private cluster by one of the big tech players, or maybe a public grid? Something like Folding@Home + Lc0 for gen AI.

*FLOPS from the TOP500 supercomputer list are measured via LINPACK benchmark in FP64 performance, double precision.

https://en.wikipedia.org/wiki/LINPACK_benchmarks
https://en.wikipedia.org/wiki/TOP500

--
Srdja
smatovic
Posts: 3030
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPU rumors 2021

Post by smatovic »

One on GPU clusters/grids...

Titan, supercomputer from 2012 to 2019 had 18,688 Nvidia K20X GPUs:
https://en.wikipedia.org/wiki/Titan_(supercomputer)

Summit, supercomputer from 2019 has 27,648 Nvidia V100 GPUs:
https://en.wikipedia.org/wiki/Summit_(supercomputer)

Folding@Home, grid, had over 650,000 GPUs in 2020, with >1 exaFLOPS (32-bit?) compute
https://archive.ph/20200412111010/https ... ome.org/os

Frontier, supercomputer from 2022 (first exaFLOPS, #2 on top500 11/2024) has 37,888 AMD MI250X GPUs:
https://en.wikipedia.org/wiki/Frontier_(supercomputer)

Colossus, supercomputer from 2024 by xAI (for the Grok AI) has currently 100,000 Nvidia HGX H100s GPUs (with estimated 100MW power usage), intended to be doubled to 200K.
https://www.tomshardware.com/desktops/s ... ts-secrets
https://www.youtube.com/watch?v=Jf8EPSBZU7Y

Lc0, grid, has currently 17 active, daily contributors ;)
https://training.lczero.org/active_users

--
Srdja
User avatar
Ras
Posts: 2662
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: GPU rumors 2021

Post by Ras »

Werewolf wrote: Sat Dec 14, 2024 1:31 pm2) Rumours online is that the M4 Ultra has been cancelled in typical Apple fashion putting profit above power users.
Are you sure with the Ultra? I thought it was the Extreme that was cancelled. Ultra is said to be two M4-Max chips, Extreme would have been four of them.
Rasmus Althoff
https://www.ct800.net
User avatar
towforce
Posts: 12124
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK
Full name: Graham Laight

Re: GPU rumors 2021

Post by towforce »

smatovic wrote: Tue Dec 17, 2024 6:33 amOne on supercomputers and FLOPS...

Very interesting - thank you.

Nice article with good examples of devices at each scale: link. The human brain is listed in the petascale section.
Want to attract exceptional people? Be exceptional.
smatovic
Posts: 3030
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPU rumors 2021

Post by smatovic »

towforce wrote: Tue Dec 17, 2024 10:29 am
smatovic wrote: Tue Dec 17, 2024 6:33 amOne on supercomputers and FLOPS...

Very interesting - thank you.

Nice article with good examples of devices at each scale: link. The human brain is listed in the petascale section.
Nice, I am not that firm with the computers from the 50s and 60s...so, maybe worth to add...

FLOPS
The first (programmable) FLOPS was achieved by the Zuse Z3 from 1941, based on electro-mechanical relays @5Hz.
https://en.wikipedia.org/wiki/Z3_(computer)

kiloFLOPS
The first (programmable) kiloFLOPS was achieved by UNIVAC I from 1951, based on vaccum-tubes @2.25MHz.
https://en.wikipedia.org/wiki/UNIVAC_I

megaFLOPS
The first megaFLOPS were achieved by transistor based mainframes like the IBM System/360 series from the 60s and 70s.
https://en.wikipedia.org/wiki/IBM_System/360

--
Srdja
Werewolf
Posts: 1936
Joined: Thu Sep 18, 2008 10:24 pm

Re: GPU rumors 2021

Post by Werewolf »

Ras wrote: Tue Dec 17, 2024 7:31 am
Werewolf wrote: Sat Dec 14, 2024 1:31 pm2) Rumours online is that the M4 Ultra has been cancelled in typical Apple fashion putting profit above power users.
Are you sure with the Ultra? I thought it was the Extreme that was cancelled. Ultra is said to be two M4-Max chips, Extreme would have been four of them.
You are quite right. It was the M4 Extreme that is rumoured to be cancelled.
smatovic
Posts: 3030
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPU rumors 2021

Post by smatovic »

Maybe a look back to "FASTRA" from 2008, the fist desktop supercomputer with 8 GPUs :)

FASTRA - Official Movie - English - Shorter Version
https://www.youtube.com/watch?v=DnIvodB2RzU

FASTRA II official movie
https://www.youtube.com/watch?v=GOpBlYx2H1o

--
Srdja
User avatar
towforce
Posts: 12124
Joined: Thu Mar 09, 2006 12:57 am
Location: Birmingham UK
Full name: Graham Laight

Re: GPU rumors 2021

Post by towforce »

smatovic wrote: Tue Dec 17, 2024 8:27 pmMaybe a look back to "FASTRA" from 2008, the fist desktop supercomputer with 8 GPUs :)

FASTRA - Official Movie - English - Shorter Version
https://www.youtube.com/watch?v=DnIvodB2RzU

That was fun - a supercomputer for €4,000 . Shame we didn't get to see the fast-rendered Bob. :cry:
Want to attract exceptional people? Be exceptional.
smatovic
Posts: 3030
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: GPU rumors 2021

Post by smatovic »

towforce wrote: Tue Dec 17, 2024 9:16 pm ...
Alice and Bob metaphors always work :)

--
Srdja