ML//transformer//attention//attention sink

An attention sink is the first token of a sequence absorbing a disproportionate amount of attention weight, regardless of its content. Even when it is a meaningless BOS token, attention heads allocate 20 to 80% of their weight to it, because softmax needs somewhere to put its spare change.

An attention sink is the first token of a sequence absorbing a disproportionate amount of attention weight, regardless of its content. Even when it is a meaningless BOS token, attention heads allocate 20 to 80% of their weight to it, because softmax needs somewhere to put its spare change.

Softmax must assign attention weights that sum to 1.0. When no token is particularly relevant to the query, the model needs somewhere to put the excess weight, and the first token becomes that default sink because it is always available (every query can attend to position 0).

The practical consequence for KV cache and long-context inference: if you use sliding window attention and evict the first token from the cache, performance drops catastrophically. The model loses its sink and attention patterns break.

StreamingLLM (2023) found the fix: always keep the first few tokens in the KV cache, even with a sliding window. With sink tokens preserved, models can process effectively infinite sequences. Without them, perplexity explodes after the context window fills.

This is an emergent behavior, not trained: nobody teaches the model to use the first token as a sink. It arises naturally from the softmax constraint plus positional encoding making position 0 a consistent anchor.

Implications for mechanistic interpretability: not all attention is semantic. Some attention patterns are just load-balancing artifacts of the softmax normalization. Reading attention weights as what the model attends to is misleading when sinks are involved.