ML//neural network//superposition

Superposition is the compression trick by which a neural network represents far more concepts than it has neurons: every neuron moonlights, representing dozens of concepts at once. Neurons represent many different features simultaneously through linear combination, and the angles between feature directions are not perfectly perpendicular.

Superposition is the compression trick by which a neural network represents far more concepts than it has neurons: every neuron moonlights, representing dozens of concepts at once. Neurons represent many different features simultaneously through linear combination, and the angles between feature directions are not perfectly perpendicular.

The more involved a neuron is across features, the more superposition it exhibits.

It makes mechanistic interpretability extremely hard, because no single neuron can be pointed at as meaning one thing. Superposition is the natural state and monosemanticity, one neuron for one concept, the aspiration; sparse autoencoders try to decompose superposed activations into clean, interpretable features.

The directions in activation space are not random: they encode structured relationships, but they overlap and interfere.