infrastructure//data center//rack

A rack is the standard steel frame, 19 inches wide between its mounting rails, that holds servers, switches and power distribution in a data center, and it is the basic unit in which the building is planned, powered and cooled. Height is counted in **rack units** (1U is 1.75 inches); a full rack is around 42 to 48U, and a server is 1U to 8U or more depending on how many GPUs it carries.

A rack is the standard steel frame, 19 inches wide between its mounting rails, that holds servers, switches and power distribution in a data center, and it is the basic unit in which the building is planned, powered and cooled. Height is counted in rack units (1U is 1.75 inches); a full rack is around 42 to 48U, and a server is 1U to 8U or more depending on how many GPUs it carries.

Above the rack the hierarchy continues, and its names are worth using exactly. Racks stand side by side in a row, usually arranged so that cold air enters the front of every machine from a cold aisle and hot air leaves into a contained hot aisle. A pod is a group of rows or racks designed as one block, with its own network spine and often its own share of the power and cooling, so it can be built, upgraded or isolated as a unit. NVIDIA's DGX SuperPOD is a vendor's packaged pod: a reference design of many GPU servers with the networking and software needed to run them as one training machine.

GPU⊂server⊂rack⊂pod⊂cluster\text{GPU} \subset \text{server} \subset \text{rack} \subset \text{pod} \subset \text{cluster}GPU⊂server⊂rack⊂pod⊂cluster

Each step of the containment changes what is fast and what is expensive: GPUs in one server talk over the fastest links, servers in one rack over short copper, racks across the hall over optical fibre (interconnect), and a training job is placed so that the chattiest parts sit closest (compute cluster).

The rack is now designed as one machine. In current AI systems such as NVIDIA's GB200 NVL72, 72 GPUs in one rack are joined by a copper backplane into a single fast domain, the rack weighs well over a tonne and draws around 120 kW, with liquid cooling and power shelves built in (power density).

Power arrives in each rack through two PDUs on separate paths, which is the physical form of the site's redundancy level at the last metre.