systems theory//Ashby's law

Ashby's law, also called the Law of Requisite Variety, is the principle of cybernetics according to which *only variety can absorb variety*: a controller must have at least as many possible responses as the environment has possible disturbances. It sets the lower bound on how rich any regulator must be, from a thermostat to a language model. W. Ross Ashby stated it in 1956, and it is the first law of cybernetics, sometimes called the most important result in systems theory.

Ashby's law, also called the Law of Requisite Variety, is the principle of cybernetics according to which only variety can absorb variety: a controller must have at least as many possible responses as the environment has possible disturbances. It sets the lower bound on how rich any regulator must be, from a thermostat to a language model. W. Ross Ashby stated it in 1956, and it is the first law of cybernetics, sometimes called the most important result in systems theory.

Formally, for a regulator RRR to control a system SSS, the variety of RRR must be greater than or equal to the variety of SSS. No clever design can circumvent this: it is an absolute constraint, like conservation of energy.

The examples scale with the problem. A thermostat (two states) controls temperature (two states: too hot, too cold). A chess engine needs astronomical variety because chess has astronomical variety, and a language model needs enormous variety because language has enormous variety.

In ML training, SFT and DPO are reactive: they redistribute existing variety from the training data but cannot increase it. RL is a feedback loop whose variety grows with compute, because the model generates novel behaviors and tests them. This is why RL discovered chain-of-thought reasoning and SFT could not.

The law explains why pretraining on diverse data is essential: the model's initial variety must be vast enough to support all downstream specialization. Narrow pretraining limits all subsequent training.