Systems Theory//Ashby's Law

W. Ross Ashby's Law of Requisite Variety (1956): "only variety can absorb variety" — a controller must have at least as many possible responses as the environment has possible disturbances.

W. Ross Ashby's Law of Requisite Variety (1956): "only variety can absorb variety" — a controller must have at least as many possible responses as the environment has possible disturbances.

The first law of cybernetics, sometimes called the most important result in systems theory.

Formal statement: for a regulator R to control a system S, the variety of R must be greater than or equal to the variety of S. No clever design can circumvent this — it is an absolute constraint, like conservation of energy.

Examples: a thermostat (2 states) controls temperature (2 states: too hot, too cold). A chess engine needs astronomical variety because chess has astronomical variety. A language model needs enormous variety because language has enormous variety.

In ML training: SFT and DPO are reactive — they redistribute existing variety from the training data but cannot increase it. RL is a feedback loop whose variety grows with compute, because the model generates novel behaviors and tests them. This is why RL discovered chain-of-thought reasoning and SFT could not.

The law explains why pretraining on diverse data is essential: the model's initial variety must be vast enough to support all downstream specialization. Narrow pretraining limits all subsequent training.