ML//model compression

Model compression is a family of techniques that reduce the storage, memory traffic or computation required by a model while trying to preserve useful capability, so that it can be deployed faster or more cheaply. Four of them recur:

Model compression is a family of techniques that reduce the storage, memory traffic or computation required by a model while trying to preserve useful capability, so that it can be deployed faster or more cheaply. Four of them recur:

Quantization stores numbers with fewer bits, without necessarily changing the architecture.

Pruning removes parameters or structures judged unnecessary; speed appears only when kernels and hardware exploit the resulting sparsity.

Distillation trains a new, usually smaller student to reproduce the behavior of a larger teacher.

Factorization replaces large tensors with cheaper structured representations.

The percentage compressed means little in isolation. What matters is whether the target runtime converts that structural change into lower latency, higher throughput or cheaper deployment without unacceptable quality loss.

Smaller on paper is only valuable when the machine notices.