ML//transformer//context window

The context window is the finite token budget available to an LLM for one generation, what RAM is to a program.

The context window is the finite token budget available to an LLM for one generation, what RAM is to a program.

Anything outside it is not directly observable. A surrounding system must retrieve, summarize or reintroduce that information, which is why RAG and context engineering matter.

For dense self-attention, attention scores grow quadratically with sequence length. Implementations such as Flash Attention reduce memory traffic without making context free, while sliding-window attention exchanges global visibility for cheaper local attention. In memory, context length and concurrent requests can make the runtime state outgrow the compressed weights (model memory footprint).

Maximum length is only part of the practical constraint: a large window can still bury the useful evidence among irrelevant tokens, so context engineering still decides which information deserves scarce attention.