ML//neural network//superposition

Every neuron moonlights — representing dozens of concepts at once, because the model has more ideas than it has neurons to store them in. Neurons represent many different features simultaneously through linear combination — angles between feature directions are not perfectly perpendicular.

Every neuron moonlights — representing dozens of concepts at once, because the model has more ideas than it has neurons to store them in. Neurons represent many different features simultaneously through linear combination — angles between feature directions are not perfectly perpendicular.

The model has far more concepts to represent than it has neurons — superposition is the compression trick.

The more involved a neuron is across features, the more superposition it exhibits.

Makes mechanistic interpretability extremely hard — you can't point at one neuron and say "this means X".

The directions in activation space aren't random — they encode structured relationships, but they overlap and interfere.