ML//parameters and weights

Parameters are the values a model learns during training. Weights are the parameter tensors that scale and combine signals, although ordinary ML conversation often uses both words almost interchangeably.

Parameters are the values a model learns during training. Weights are the parameter tensors that scale and combine signals, although ordinary ML conversation often uses both words almost interchangeably.

Parameter count estimates model storage and some compute, but it does not equal total inference memory. Activations, runtime workspaces and the KV cache may dominate under large batches or long contexts. Nor does a larger parameter count guarantee a better model; architecture, data and training determine what those numbers encode.

Weights are frozen experience. Context is today's paperwork.