ML//transformer//context window

A program has RAM; an LLM inference has a **context window**: the finite token budget available to the model for that generation.

A program has RAM; an LLM inference has a context window: the finite token budget available to the model for that generation.

Anything outside it is not directly observable. A surrounding system must retrieve, summarize or reintroduce that information, which is why RAG and context engineering matter.

For dense self-attention, attention scores grow quadratically with sequence length. Implementations such as Flash Attention reduce memory traffic without making context free, while sliding-window attention exchanges global visibility for cheaper local attention.

The practical constraint is not merely maximum length. A large window can still bury the useful evidence among irrelevant tokens.