hardware//roofline model

The roofline model is a performance bound that says how fast a computation can run on a given processor from two hardware numbers and one property of the algorithm, and it is used to tell at a glance whether a workload is limited by arithmetic or by memory, and so which upgrade or rewrite could speed it up. It was proposed by Williams, Waterman and Patterson in 2009 and is the standard first sketch when sizing hardware for a filter, a network or a simulation.

The roofline model is a performance bound that says how fast a computation can run on a given processor from two hardware numbers and one property of the algorithm, and it is used to tell at a glance whether a workload is limited by arithmetic or by memory, and so which upgrade or rewrite could speed it up. It was proposed by Williams, Waterman and Patterson in 2009 and is the standard first sketch when sizing hardware for a filter, a network or a simulation.

The algorithm's property is its arithmetic intensity III: the number of floating-point operations it performs per byte it brings from memory. The bound is

performance=min⁡(peak FLOP/s,  I×memory bandwidth).\text{performance}=\min\big(\text{peak FLOP/s},\; I\times\text{memory bandwidth}\big).performance=min(peak FLOP/s,I×memory bandwidth).

Plotted against III on log axes, it is a sloped line (memory sets the pace) that meets a flat roof (arithmetic sets the pace) at the ridge point, the intensity at which the two balance.

Matrix-vector work sits low on the slope. Multiplying an n×nn\times nn×n matrix by a vector reads every element once for two operations, about 0.5 FLOP per byte in 32-bit floats, so a filter update, a linear layer at batch size one or one token of language-model generation runs at a small fraction of a GPU's peak, waiting for memory bandwidth (memory-bound).

Matrix-matrix work sits under the roof. Multiplying two n×nn\times nn×n matrices does about 2n32n^32n3 operations on 3n23n^23n2 numbers, so intensity grows with nnn and large products reach peak arithmetic (compute-bound); that is why training with big batches loves a GPU.

The model tells you which lever works. Below the ridge, a faster chip with the same memory buys nothing, while fewer bytes do: quantization or batching raise the useful work per byte. Above it, only more arithmetic throughput helps.

It ignores latency, caches that hold the working set, and control flow, so it bounds and never predicts: a branch-heavy estimator on an MCU can fall far below both lines. Its value is the first-order question it forces before anyone buys hardware (embedded system).