NVIDIA//A100

- The GPU that trained GPT-3, PaLM, and most 2021-2023 LLMs.

The GPU that trained GPT-3, PaLM, and most 2021-2023 LLMs.

80GB HBM2e, Tensor Cores for mixed-precision training.

Clusters of thousands defined the scaling era. ~$10K-$15K per card.