mathematics//statistics//variance

The **variance** quantifies dispersion as the mean squared deviation from the mean, and it is not to be confused with useful information. For a random variable \(X\) with finite second moment it is defined as follows:

The variance quantifies dispersion as the mean squared deviation from the mean, and it is not to be confused with useful information. For a random variable XXX with finite second moment it is defined as follows:

μ=E[X],Var⁡(X)=E[(X−μ)2]=σ2.\mu=\mathbb E[X],\qquad \operatorname{Var}(X)=\mathbb E[(X-\mu)^2]=\sigma^2.μ=E[X],Var(X)=E[(X−μ)2]=σ2.

The standard deviation σ=Var⁡(X)\sigma=\sqrt{\operatorname{Var}(X)}σ=Var(X)​ returns the units of XXX: for a temperature, variance is in degrees squared and standard deviation in degrees.

Read literally, it takes how far each value is from the mean, squares that distance and averages the squares. Each squared deviation is the area of a square, so the variance is the mean area of those squares and the standard deviation the side of a square with that area.

mean x̄4.52 variance s²3.58 σ = √s²1.89 Each deviation from the mean as a square: the variance is the mean area of those squares, and the dashed square, with that area, has side σ.

Drag a point away from the others and watch its square take over the sum: a point twice as far adds four times the area.

?Why squared, and not the absolute value?

It looks arbitrary and is not. The variances of independent things add, Var⁡(A+B)=Var⁡(A)+Var⁡(B)\operatorname{Var}(A+B)=\operatorname{Var}(A)+\operatorname{Var}(B)Var(A+B)=Var(A)+Var(B): Pythagoras for noise, independent errors behave like perpendicular vectors, and with absolute values this does not happen. The square is also differentiable, so it can be optimized (least squares, and a Kalman filter minimizes exactly a mean squared error), and it is the parameter of the normal distribution. The price is the one the figure shows: a value ten times further weighs a hundred times more.

?Divide by nnn or by n−1n-1n−1?

By n−1n-1n−1 when the mean was also taken from the data (Bessel's correction). The sample mean sits a little closer to its own data than the true mean does, so the deviations come out slightly small, and dividing by n−1n-1n−1 compensates. With 10,000 values it makes no difference; with 5, it does. Python trap: np.var(x) divides by nnn by default (ddof=0), but pandas.Series.var() divides by n−1n-1n−1 (ddof=1).

The variance measures dispersion and says nothing of shape. Two distributions can share mean and variance and still differ completely: symmetric or skewed, unimodal or clustered, with thin or heavy tails.

The standard deviation is for thinking and the variance for computing. A GPS with σ=2\sigma=2σ=2 m has a variance of 4 square metres of error, and nobody in the history of humanity has imagined a square metre of error. For a normal, σ=5\sigma=5σ=5 m means two readings in three fall within 5 m of the truth and nineteen in twenty within 10 m (normal distribution). The variance is what one computes with, because variances of independent errors add; a Kalman filter works in variances throughout. A smaller σ\sigmaσ means a more precise sensor, not necessarily a more exact one (precision and accuracy).

The sample version estimates both from data: for x1,…,xNx_1,\ldots,x_Nx1​,…,xN​, xˉ=1N∑ixi\bar x=\frac1N\sum_i x_ixˉ=N1​∑i​xi​ and s2=1N−1∑i(xi−xˉ)2s^2=\frac1{N-1}\sum_i(x_i-\bar x)^2s2=N−11​∑i​(xi​−xˉ)2, with Bessel's divisor. On a microcontroller the running version is Welford's algorithm: the shortcut E[x2]−E[x]2\mathbb E[x^2]-\mathbb E[x]^2E[x2]−E[x]2 subtracts two nearly equal large numbers and returns garbage in float32, sometimes negative.

Variance is not information. PCA maximizes retained variance, and calling that information abbreviates a geometric criterion; it equals neither Shannon entropy nor relevance for a decision. A noisy sensor has high variance, while a small difference between two sensors can reveal a fault (redundant sensors computes a case where the lowest-variance mode keeps exactly that discrepancy).- covariance studies joint variation; the covariance matrix generalizes variance to any direction; projection defines what can be lost when compressing.