mathematics//statistics//covariance
Covariance is a statistic of two variables that measures how their deviations from their means align, and it is used to say whether two quantities tend to move together; it has to be told apart from normalized association and from causality. The **covariance** is
Covariance is a statistic of two variables that measures how their deviations from their means align, and it is used to say whether two quantities tend to move together; it has to be told apart from normalized association and from causality. The covariance is
Cov(X,Y)=E[(X−μX)(Y−μY)].\operatorname{Cov}(X,Y)=\mathbb E[(X-\mu_X)(Y-\mu_Y)].Cov(X,Y)=E[(X−μX)(Y−μY)].
If both deviations tend to share a sign it is positive; if they tend to oppose, negative. Its magnitude depends on the scales and its units are [X][Y][X][Y][X][Y]. Variance is the special case Cov(X,X)\operatorname{Cov}(X,X)Cov(X,X): the covariance of a variable with itself. From a sample of nnn pairs it is estimated as
sXY=1n−1∑i=1n(xi−xˉ)(yi−yˉ).s_{XY}=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar x)(y_i-\bar y).sXY=n−11i=1∑n(xi−xˉ)(yi−yˉ).
Just as variance is the mean area of squares, covariance is the mean area of signed rectangles: each point draws a rectangle to the mean point, counted positive when its two deviations share a sign and negative when they oppose. Neither is more useful in general: covariance answers a question about two variables, variance about one.
covariance3.36 ρ0.91 Each point spans a rectangle to the mean point; its area is the product of the two deviations, copper when they share a sign and steel when they oppose. The covariance is the mean signed area.
Drag a point across the mean lines and watch its rectangle change colour, then try the y=x2y=x^2y=x2 preset, where the copper and steel areas cancel to zero.
Pearson correlation. If both standard deviations are finite and nonzero,
ρXY=Cov(X,Y)σXσY,−1≤ρXY≤1.\rho_{XY}=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y},\qquad -1\le\rho_{XY}\le1.ρXY=σXσYCov(X,Y),−1≤ρXY≤1.
ρ\rhoρ is a dimensionless coefficient: it removes positive changes of unit and summarizes linear association. Flipping the orientation of one axis flips its sign. If one variable is constant the expression is undefined; it must not be replaced by zero automatically.
A joint distribution is required. A scatter plot shows observed pairs (xi,yi)(x_i,y_i)(xi,yi) from the same case, object or comparable instant. Knowing only the marginal of XXX and the marginal of YYY does not fix their covariance: one needs to know how they pair up. Two temperature sensors read at the same time give pairs; two lists with no established correspondence do not define the same statistical question.
Zero does not mean independence. Let XXX be uniform on [−1,1][-1,1][−1,1] and Y=X2Y=X^2Y=X2. By symmetry E[X]=E[X3]=0\mathbb E[X]=\mathbb E[X^3]=0E[X]=E[X3]=0, so Cov(X,Y)=0\operatorname{Cov}(X,Y)=0Cov(X,Y)=0, yet YYY is fully determined by XXX. Zero correlation rules out that linear association, not every dependence. Independence implies zero covariance when the moments exist; the converse needs extra hypotheses.
Association does not determine mechanisms. Temperature and electricity consumption can covary through common demand, direct causation, control or omitted variables; summer heat raises both a compressor's load and its failure rate, and in a regulated oven feedback hides the heater's effect on temperature almost entirely (causal inference). A covariance is not the coupling coefficient of a dynamic equation, and a statistical network built from correlations must declare that construction (adjacency and degree).
Applied to errors, covariance says how two doubts relate, and that is where the information for fusion lives: a filter that knows its position and velocity errors covary can correct the velocity from a position reading alone. The covariance matrix gathers many of these associations at once.