ML//alignment//mechanistic interpretability//circuit

A circuit is a minimal subgraph of a neural network that implements a specific behavior: the exact set of neurons, attention heads and MLP layers that collaborate to produce a particular computation.

A circuit is a minimal subgraph of a neural network that implements a specific behavior: the exact set of neurons, attention heads and MLP layers that collaborate to produce a particular computation.

It is the core unit of mechanistic interpretability. Instead of asking what the model does as a whole, the study of circuits asks which exact components do one specific thing, and how they connect.

The induction head circuit is the standard example. Two attention heads work together: the first (previous token head) copies position information, the second (induction head) uses it to predict the next token in a repeated sequence. This is a circuit: specific components, specific wiring, specific behavior.

Finding circuits means ablation: components are removed or corrupted and what breaks is measured. If deactivating three attention heads kills the model's ability to do indirect object identification and nothing else, those heads are the circuit for that task.

Circuits can be composed: simple circuits combine into more complex behaviors. The superposition problem is that circuits overlap; the same neurons participate in multiple circuits, making them hard to isolate.

This is why sparse autoencoders matter: they decompose superposition into individual features, making it possible to trace which features activate which circuits. SAEs are the microscope, circuits are what you find under it.

The residual stream is the communication bus: circuits write to it and read from it. Understanding circuits means understanding what information flows through the residual stream at each layer.