ML//neural network//monosemanticity

Monosemanticity is the holy grail of interpretability: one neuron, one meaning, with no double shifts, no moonlighting, just clean signal. It is the opposite of superposition, and a key goal of mechanistic interpretability, since extracting monosemantic features would show what the model actually knows.

Monosemanticity is the holy grail of interpretability: one neuron, one meaning, with no double shifts, no moonlighting, just clean signal. It is the opposite of superposition, and a key goal of mechanistic interpretability, since extracting monosemantic features would show what the model actually knows.

The landmark paper is Towards Monosemanticity: Decomposing Language Models with Dictionary Learning (Anthropic, 2023), which uses sparse autoencoders to decompose activations into a larger set of interpretable features. Its insight is that the model does have clean features: they are encoded as directions in superposed space rather than as individual neurons.