ML//neural network//sparse autoencoder

A sparse autoencoder is a tool for extracting interpretable features from superposed neural activations, used in mechanistic interpretability research to peer inside transformer layers. Its architecture runs in three steps:

A sparse autoencoder is a tool for extracting interpretable features from superposed neural activations, used in mechanistic interpretability research to peer inside transformer layers. Its architecture runs in three steps:

1The encoder expands the activationsinto a much larger, overcomplete basis2A sparsity constraint forces most dimensions to zero3The decoder reconstructs the activations

The non-zero dimensions in the expanded space correspond to active features, ideally monosemantic ones. This is dictionary learning: the autoencoder learns a dictionary of features, and each activation is a sparse combination of dictionary entries.