ML//inference//sampling//temperature

Temperature is a sampling parameter of text generation that scales the logits (the raw pre-normalization scores over the vocabulary) before the softmax, dividing them by \(T\). It is the simplest generation knob: low temperature suits factual tasks, high temperature creative ones.

Temperature is a sampling parameter of text generation that scales the logits (the raw pre-normalization scores over the vocabulary) before the softmax, dividing them by TTT. It is the simplest generation knob: low temperature suits factual tasks, high temperature creative ones.

At T=0T=0T=0 decoding is greedy and always picks the highest probability; T=1T=1T=1 keeps the raw distribution; T>1T>1T>1 makes it more random.

A high TTT flattens the distribution (more creative) and a low TTT sharpens it (more confident).