systems theory//cybernetics

Cybernetics is the science of control and communication in systems, the control-focused branch of systems theory, founded by Norbert Wiener in 1948, the same year as Shannon's information theory. Its core concept is the feedback loop: a system acts, observes the result, and adjusts its next action. Negative feedback stabilizes (thermostats, PID controllers, KL divergence in RL); positive feedback amplifies (compounding errors, reward hacking, viral growth). Its founding constraint is Ashby's law: a regulator needs at least as much variety as the disturbances it must absorb, whatever its design.

Cybernetics is the science of control and communication in systems, the control-focused branch of systems theory, founded by Norbert Wiener in 1948, the same year as Shannon's information theory. Its core concept is the feedback loop: a system acts, observes the result, and adjusts its next action. Negative feedback stabilizes (thermostats, PID controllers, KL divergence in RL); positive feedback amplifies (compounding errors, reward hacking, viral growth). Its founding constraint is Ashby's law: a regulator needs at least as much variety as the disturbances it must absorb, whatever its design.

The engineering practice is older than the name. Watt's centrifugal governor (1788) held a steam engine at speed by feeding the speed back to the steam valve, and James Clerk Maxwell's On Governors (1868) was the first mathematical analysis of why such governors sometimes hunted, oscillating instead of settling: the stability question every closed-loop system still has to answer. Harold Black's negative-feedback amplifier (1927) used the same trick to make drifting vacuum tubes precise, trading gain for insensitivity to the parts, which is the bargain of feedback control.

Machine-learning training reads naturally as a feedback system:

SFT and DPO are open-loop: the dataset is fixed and no output of the model enters the loop during training.

RL (PPO, GRPO) is closed-loop: the model generates, gets feedback and adapts, a textbook cybernetic system.

Reward hacking is positive feedback gone wrong: the model exploits the reward signal and amplifies degenerate behaviors.

KL divergence acts as negative feedback. It keeps the RL policy from drifting too far from the reference model, a stabilizing constraint like a governor on a steam engine.

Second-order cybernetics studies systems that observe themselves observing. Constitutional AI is an example: the model critiques its own outputs using principles, then trains on the critique, so the observer is part of the system.