ML//AI progress//effective compute
Effective compute is the amount of compute a training run would have needed with a fixed earlier algorithm to reach the same performance, that is, physical compute multiplied by the efficiency gained from better algorithms since then, and it is used to put hardware and algorithmic progress on one scale. A run that uses ten times the hardware with algorithms three times as efficient counts as thirty times the effective compute.
Effective compute is the amount of compute a training run would have needed with a fixed earlier algorithm to reach the same performance, that is, physical compute multiplied by the efficiency gained from better algorithms since then, and it is used to put hardware and algorithmic progress on one scale. A run that uses ten times the hardware with algorithms three times as efficient counts as thirty times the effective compute.
Its growth is a product of factors, each with its own clock:
geff=gspend×gFLOP/$×galgg_{\text{eff}} = g_{\text{spend}} \times g_{\text{FLOP}/\$} \times g_{\text{alg}}geff=gspend×gFLOP/$×galg
spending on compute, compute per dollar of hardware, and algorithmic progress. For frontier language models the physical part has grown about five times a year since 2020, mostly from spending, with hardware price-performance adding roughly 1.4 to 1.5 times a year.
Because the factors multiply, slowing one is enough to slow the product, and the factor that is hardest to keep growing (money, chip packaging, power) sets the pace (rate-limiting step).
Algorithmic gains are measured at some reference scale, and they can be much larger at frontier scale than in small ablations, so the effective-compute number carries the uncertainty of its algorithmic factor.