ML//transformer//tokenizer//chat template

A chat template is the rule that turns a list of chat messages (system, user, assistant, tool) into the single string of text a language model actually reads, marking each turn with special tokens. It exists because the model only continues text: roles, tools and the structure of a conversation reach it as tokens in one sequence. The template ships with the model; inside a GGUF file it is the metadata key `tokenizer.chat_template`, a small Jinja program the engine runs on the messages.

A chat template is the rule that turns a list of chat messages (system, user, assistant, tool) into the single string of text a language model actually reads, marking each turn with special tokens. It exists because the model only continues text: roles, tools and the structure of a conversation reach it as tokens in one sequence. The template ships with the model; inside a GGUF file it is the metadata key tokenizer.chat_template, a small Jinja program the engine runs on the messages.

Qwen models use the ChatML layout. Each message opens with <|im_start|>, the role and a newline, and closes with <|im_end|>; the prompt ends with an open turn, <|im_start|>assistant, from which the model writes its answer. A system prompt, a user question and that open turn are therefore one continuous text of a few lines, and the model's job is to continue it.

The system prompt is only the beginning of that text, which is why it counts toward the context window and why changing it invalidates prefix caching for everything after it.

A wrong template (another family's, or one that drops a token) degrades the answers without raising any error; engines read the template from the model file to avoid it.

Tool definitions, JSON schemas and images all end up as tokens in the same string, so they cost prefill like any other text.