hardware//I/O//PCIe

PCIe (PCI Express) is the point-to-point serial link that connects a computer's CPU to its expansion devices (graphics cards, NVMe SSDs, network cards), and in an AI machine it is the road by which a model travels from host RAM to the GPU's own memory. A link is a bundle of **lanes**, each lane a pair of differential wire pairs, one for each direction; a graphics card takes 16 lanes (x16), an SSD usually 4.

PCIe (PCI Express) is the point-to-point serial link that connects a computer's CPU to its expansion devices (graphics cards, NVMe SSDs, network cards), and in an AI machine it is the road by which a model travels from host RAM to the GPU's own memory. A link is a bundle of lanes, each lane a pair of differential wire pairs, one for each direction; a graphics card takes 16 lanes (x16), an SSD usually 4.

Each generation doubles the rate per lane. A PCIe 4.0 x16 slot carries about 32 GB/s in each direction and a PCIe 5.0 x16 slot about 64 GB/s; an NVMe SSD on four PCIe 4.0 lanes reads at about 7 GB/s. The transfers themselves are DMA: the device copies to or from RAM without the CPU touching each byte, at full speed only from page-locked (pinned) host memory.

PCIe is crossed at load time, not at every step.

Copying a 14 GB model (7 billion weights in 16 bits) from host RAM to an H100 over PCIe 5.0 takes about 220 ms, once. From then on the GPU reads the weights from its HBM at 3.35 TB/s, 4.2 ms per step. A model left in host RAM and pulled across PCIe at every step (offloading) pays the 220 ms for every token, about fifty times slower.

When a model does not fit in VRAM, local engines usually avoid offloading over the link: they run the layers that do not fit on the CPU, from host RAM, and send only activations across PCIe.

Traffic between GPUs inside one server goes over NVLink instead, at 900 GB/s per H100 (interconnect); PCIe stays the link to the host and its disks.

Its place among the other links of a machine is in memory hierarchy.