Systems Theory//cybernetics

The science of control and communication in systems — founded by Norbert Wiener (1948), same year as Shannon's information theory.

The science of control and communication in systems — founded by Norbert Wiener (1948), same year as Shannon's information theory.

Core concept: the feedback loop. A system acts, observes the result, and adjusts its next action. Negative feedback stabilizes (thermostats, PID controllers, KL divergence in RL). Positive feedback amplifies (compounding errors, reward hacking, viral growth)

Ashby's Law is the first law of cybernetics: a controller must match the variety of its environment.

ML as feedback systems

SFT, DPO: open-loop (no feedback during training — fixed dataset, no model output in the loop)

RL (PPO, GRPO): closed-loop (the model generates, gets feedback, adapts — textbook cybernetic system)

Reward hacking: positive feedback gone wrong — the model exploits the reward signal, amplifying degenerate behaviors.

KL divergence as negative feedback: prevents the RL policy from diverging too far from the reference model — a stabilizing constraint, like a governor on a steam engine.

Second-order cybernetics: systems that observe themselves observing. Constitutional AI is an example — the model critiques its own outputs using principles, then trains on the critique. The observer is part of the system.