hardware//GPU

A GPU is a massively parallel processor designed to apply similar operations across large collections of data. Graphics supplied the original workload; matrix-heavy ML training and inference fit much of the same parallel machinery.

A GPU is a massively parallel processor designed to apply similar operations across large collections of data. Graphics supplied the original workload; matrix-heavy ML training and inference fit much of the same parallel machinery.

Thousands of simple cores executing the same instruction on different data (SIMD/SIMT)

Originally designed for rendering, now critical for ML training and inference.

Available as discrete add-in cards or integrated units sharing the SoC die with the CPU

The useful device is larger than its transistor diagram. Drivers, compilers, kernels, libraries and memory systems determine whether an application can reach theoretical throughput. In AI, NVIDIA's advantage has historically included this software ecosystem as well as silicon.

A GPU wins by making one kind of crowd move together. Irregular sequential work still belongs elsewhere.