ML//Transformer//logits

Raw scores over the entire vocabulary before normalization: Wu × normalized_output = logits.

Raw scores over the entire vocabulary before normalization: Wu × normalized_output = logits.

Each value = dot product between the context vector and one row of the LM head — measures "how aligned is this output with each possible next token".

Not probabilities — can be negative, arbitrarily large. Softmax converts them to a proper distribution.

Temperature divides logits before softmax: logits/T. Low T = peaky distribution (confident), high T = flat (creative)

In DPO: the model computes logits for both preferred and rejected outputs, widening the gap between them.