ML//scaling laws

Kaplan et al. (2020): performance follows power laws in compute, data, and parameters.

Kaplan et al. (2020): performance follows power laws in compute, data, and parameters.

Predictable — you can forecast capability from training budget before running the experiment.

Changed how labs plan: don't architect smarter, just scale bigger. Then Chinchilla refined the recipe.