mathematics//statistics//covariance

2026-09-06

How to tell joint variation, normalized association and causality apart. The **covariance** measures how deviations from the means align,

How to tell joint variation, normalized association and causality apart. The covariance measures how deviations from the means align,

Cov⁡(X,Y)=E[(X−μX)(Y−μY)].\operatorname{Cov}(X,Y)=\mathbb E[(X-\mu_X)(Y-\mu_Y)].Cov(X,Y)=E[(X−μX)(Y−μY)].

If both deviations tend to share a sign it is positive; if they tend to oppose, negative. Its magnitude depends on the scales and its units are [X][Y][X][Y][X][Y]. Variance is the special case Cov⁡(X,X)\operatorname{Cov}(X,X)Cov(X,X). Covariance is not "more useful" in general: it answers a question about two variables, variance about one.

Pearson correlation. If both standard deviations are finite and nonzero,

ρXY=Cov⁡(X,Y)σXσY,−1≤ρXY≤1.\rho_{XY}=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y},\qquad -1\le\rho_{XY}\le1.ρXY=σXσYCov(X,Y),−1≤ρXY≤1.

ρ\rhoρ is a dimensionless coefficient: it removes positive changes of unit and summarizes linear association. Flipping the orientation of one axis flips its sign. If one variable is constant the expression is undefined; it must not be replaced by zero automatically.

A joint distribution is required. A scatter plot shows observed pairs (xi,yi)(x_i,y_i)(xi,yi) from the same case, object or comparable instant. Knowing only the marginal of XXX and the marginal of YYY does not fix their covariance: one needs to know how they pair up. Two temperature sensors read at the same time give pairs; two lists with no established correspondence do not define the same statistical question.

Zero does not mean independence. Let XXX be uniform on [−1,1][-1,1][−1,1] and Y=X2Y=X^2Y=X2. By symmetry E[X]=E[X3]=0\mathbb E[X]=\mathbb E[X^3]=0E[X]=E[X3]=0, so Cov⁡(X,Y)=0\operatorname{Cov}(X,Y)=0Cov(X,Y)=0, yet YYY is fully determined by XXX. Zero correlation rules out that linear association, not every dependence. Independence implies zero covariance when the moments exist; the converse needs extra hypotheses.

Association does not determine mechanisms. Temperature and electricity consumption can covary through common demand, direct causation, control or omitted variables. A covariance is not the coupling coefficient of a dynamic equation, and a statistical network built from correlations must declare that construction (adjacency and degree).

The covariance matrix gathers many of these associations at once.