GPU rumors 2021

[smatovic]

[...]
Not sure what Facebook and Tesla are doing on the list? I'm guessing that Amazon is on the list because it's offering generative AI via AWS? Also, of course, the Echo - the first smart speaker.

Facebook works on generative AI, the Metaverse did not work out very well, maybe now AI agents, Amazon is in too, not only via AWS, and I can imagine that one day Elon Musk will set up a parent company for all his Xes, combine them together, and it is pretty impressive, or alike.

Shortcut, it is about tech player market cap, and how much you can invest in AI, all seven have own silicon running for AI.

--
Srdja

[smatovic]

[...]
Generative AI is really taking off these days, now it is about the so called "Magnificent Seven":

https://stockanalysis.com/list/magnificent-seven/
https://stockanalysis.com/stocks/nvda/market-cap/

Apple, Microsoft, Google, Amazon, Facebook, Tesla and Nvidia.

I like to call them the seven heads of the beast

--
Srdja

To make the list complete, the 'Magnificent Seven' with own AI silicon:

[...]
Amazon AWS Graviton CPU + Inferentia + Trainium AI accelerators:
https://en.wikipedia.org/wiki/AWS_Graviton
https://www.aboutamazon.com/news/aws/wh ... -anthropic

Microsoft Cobalt CPU + Maia AI accelerator:
https://www.tomshardware.com/news/micro ... cpu-custom

Google Axion CPU + TPU AI accelerator:
https://cloud.google.com/blog/products/ ... -based-cpu
https://en.wikipedia.org/wiki/Tensor_Processing_Unit

Interesting game IMO, Intel, AMD, Nvidia vs. Amazon, Microsoft, Google, dunno
what IBM and Oracle have in hands in regard of AI accelerators, there are still
the POWER10 and SPARC processors. Still wait to see a breakthrough with RISC-V
chips in consumer and datacenter markets.

See also:
China boosts in silicon...
viewtopic.php?t=72566
[..]

Apple To Power AI Tools With In-House Server Chips This Year
https://apple.slashdot.org/story/24/05/ ... -this-year

Facebook - Introducing Our Next Generation Infrastructure for AI
https://about.fb.com/news/2024/04/intro ... re-for-ai/

Tesla - FSD chip/ Dojo chip
https://www.tesla.com/AI

Nvidia - data center products
https://www.nvidia.com/en-us/data-center/

--
Srdja

[smatovic]

[...]
It has now come out the about to be released Threadripper 96 core CPU maxes out at 3.2 GHz all core turbo - 200 MHz slower than the top EPYC - so it'll be interesting to see what OS Nvidia's CPU runs on.

Ubuntu Linux 22.04 server LTS (aarch64) runs on Nvidia Grace Hopper CPUs:

https://research.colfax-intl.com/ubuntu ... superchip/
https://docs.nvidia.com/grace-ubuntu-install-guide.pdf

Idk if its already arm v9 ISA optimized.

--
Srdja

[Werewolf]

[...]
It has now come out the about to be released Threadripper 96 core CPU maxes out at 3.2 GHz all core turbo - 200 MHz slower than the top EPYC - so it'll be interesting to see what OS Nvidia's CPU runs on.

Ubuntu Linux 22.04 server LTS (aarch64) runs on Nvidia Grace Hopper CPUs:

https://research.colfax-intl.com/ubuntu ... superchip/
https://docs.nvidia.com/grace-ubuntu-install-guide.pdf

Idk if its already arm v9 ISA optimized.

--
Srdja

Slightly below top-end EPYC in the benchmarks IIRC?

[smatovic]

Idk, there is a 72 core ARM v8 Neoverse v2 with SF 16.1, dunno if this is Grace:

https://www.phoronix.com/review/nvidia- ... -benchmark

Here some Phoronix benchmarks:

https://www.phoronix.com/review/nvidia- ... readripper
https://www.phoronix.com/review/nvidia- ... -benchmark

Still no Lc0 benchmark numbers in the web.

But I can imagine such a box would be a nice fit for your desk

https://gptshop.ai/pics/pics.html
https://gptshop.ai/config/indexus.html

--
Srdja

[smatovic]

Idk, there is a 72 core ARM v8 Neoverse v2 with SF 16.1, dunno if this is Grace:
...

Wrong link, here:

https://openbenchmarking.org/test/pts/stockfish

Code: Select all

ARMv8 Neoverse-V2 72-Core 84th 3 84036179

--
Srdja

[smatovic]

One on supercomputers:

The US has two exaFLOPS* supercomputers running, and one in pipe:

Frontier, with AMD Epyc CPUs and MI250X GPUs:
https://en.wikipedia.org/wiki/Frontier_(supercomputer)

Aurora, with Intel Max CPUs and Max GPUs:
https://en.wikipedia.org/wiki/Aurora_(supercomputer)

El Capitan, with AMD Epyc CPUs and MI300A APUs:
https://en.wikipedia.org/wiki/El_Capita ... rcomputer)

Rumor has it that China runs already three exaFLOPS systems not listed on TOP500:

- Sugon, with Hygon x86-64 CPUs+DCUs
- Tianhe 3, with Matrix-3000 accelerators
- OceanLight, with ShenWei 26010p processors

And Europe is now building its first exaFLOPS system, with Nvidia Grace Hopper superchip and also Rhea, an European ARM CPU from EPI, eruopean processor initiative, "Jupiter":
https://www.fz-juelich.de/en/ias/jsc/jupiter

The US and European systems are all in the ~500 millon dollar range.

I myself am keen to see benchmarks of the WIP RISC-V CPU from EPI with accelerator modules:
https://www.european-processor-initiati ... celerator/

Top500 06/2024:
https://www.top500.org/lists/top500/2024/06/

*supercomputer FLOPS from TOP500 list are measured in FP64 Linpack Benchmark performance:
https://en.wikipedia.org/wiki/LINPACK_benchmarks

--
Srdja

[smatovic]

....on Chinese GPUs:

Moore Threads
https://en.wikipedia.org/wiki/Moore_Threads

Biren Technology
https://en.wikipedia.org/wiki/Biren_Technology

Zhaoxin
https://en.wikipedia.org/wiki/Zhaoxin#Discrete_GPU

--
Srdja

Lingjiu GP201 compares to Nvidia lowest end model GT 1030 by specs:

The GP201 boasts a base clock of 1.2 GHz, single precision floating point performance of 1.2 TFLOPS, 2GB DDR4 VRAM, and power consumption of up to 30W.

Chinese-made GPU beats performance of 10-year-old integrated AMD graphics — Lingjiu GP201 hits mass production
https://www.tomshardware.com/pc-compone ... production

I can imagine that this is just a beginning, or alike.

--
Srdja

[smatovic]

[...]
Interesting game IMO, Intel, AMD, Nvidia vs. Amazon, Microsoft, Google, dunno
what IBM and Oracle have in hands in regard of AI accelerators, there are still
the POWER10 and SPARC processors. Still wait to see a breakthrough with RISC-V
chips in consumer and datacenter markets.
[...]

As for CPUs, IBM has still the POWER10 and Z-series in offer:
https://en.wikipedia.org/wiki/Power10
https://www.ibm.com/power?lnk=flatitem
https://en.wikipedia.org/wiki/IBM_Z#IBM_z16
https://en.wikipedia.org/wiki/IBM_Telum

both Z-series (Telum) and Power10 have matrix AI acceleration.

Fujitsu keeps the SPARC64 XII till 2029 for sale:
https://en.wikipedia.org/wiki/SPARC
https://en.wikipedia.org/wiki/SPARC64_V#SPARC64_XII
https://www.fujitsu.com/global/products ... nix/sparc/

and looks like Oracle switched to ARM based Ampere Altra:
https://en.wikipedia.org/wiki/Ampere_Computing

Ampere Altra available as 1 and 2 sockets systems, with up to 128 cores and 128 PCIe lanes per socket:
https://www.deltacomputer.com/server/st ... altra.html

And, there is ARM SME, scalable matrix extensions, for arm v9a, AI matrix operations on CPU:
https://community.arm.com/arm-community ... chitecture

Interesting times.

--
Srdja

[Werewolf]

[...]
Interesting game IMO, Intel, AMD, Nvidia vs. Amazon, Microsoft, Google, dunno
what IBM and Oracle have in hands in regard of AI accelerators, there are still
the POWER10 and SPARC processors. Still wait to see a breakthrough with RISC-V
chips in consumer and datacenter markets.
[...]

As for CPUs, IBM has still the POWER10 and Z-series in offer:
https://en.wikipedia.org/wiki/Power10
https://www.ibm.com/power?lnk=flatitem
https://en.wikipedia.org/wiki/IBM_Z#IBM_z16
https://en.wikipedia.org/wiki/IBM_Telum

both Z-series (Telum) and Power10 have matrix AI acceleration.

Fujitsu keeps the SPARC64 XII till 2029 for sale:
https://en.wikipedia.org/wiki/SPARC
https://en.wikipedia.org/wiki/SPARC64_V#SPARC64_XII
https://www.fujitsu.com/global/products ... nix/sparc/

and looks like Oracle switched to ARM based Ampere Altra:
https://en.wikipedia.org/wiki/Ampere_Computing

Ampere Altra available as 1 and 2 sockets systems, with up to 128 cores and 128 PCIe lanes per socket:
https://www.deltacomputer.com/server/st ... altra.html

And, there is ARM SME, scalable matrix extensions, for arm v9a, AI matrix operations on CPU:
https://community.arm.com/arm-community ... chitecture

Interesting times.

--
Srdja

How fast is SPARC these days compared with Intel/AMD? We never see Stockfish benchmarks on them, so I suppose they are low-power cloud centre CPUs?


I remember in the 1990s driving into London on the M4 and they had a giant advert on the side of a building that advertised PCs with SPARC CPUs. It read "SPARCS fly"