ML//unsupervised learning//PCA//redundant sensors
Worked example: when a low-variance direction is the most important one for diagnosis. Two sensors measure a common temperature,
Worked example: when a low-variance direction is the most important one for diagnosis. Two sensors measure a common temperature,
X1=20+S+η1,X2=20+S+η2.X_1=20+S+\eta_1,\qquad X_2=20+S+\eta_2.X1=20+S+η1,X2=20+S+η2.
SSS is process variation with zero mean and variance 4. The noises η1,η2\eta_1,\eta_2η1,η2 have zero mean and variance 1, independent of each other and of SSS. Temperatures are in degrees Celsius, variances in degrees squared.
What the pairs say. Each sensor has variance 4+1=54+1=54+1=5; their covariance is 4 through the shared signal. So
Σ=[5445],ρ12=45=0.8.\Sigma=\begin{bmatrix}5&4\\4&5\end{bmatrix},\qquad \rho_{12}=\tfrac45=0.8.Σ=[5445],ρ12=54=0.8.
The covariance has units, the correlation does not. The positive correlation expresses joint variation under the model; it does not by itself prove the shared cause we assumed (covariance).
Two directions with meaning. v+=(1,1)T/2v_+=(1,1)^{\mathsf T}/\sqrt2v+=(1,1)T/2 with λ+=9\lambda_+=9λ+=9, and v−=(1,−1)T/2v_-=(1,-1)^{\mathsf T}/\sqrt2v−=(1,−1)T/2 with λ−=1\lambda_-=1λ−=1 (covariance matrix). The common coordinate a+=(X1+X2−40)/2a_+=(X_1+X_2-40)/\sqrt2a+=(X1+X2−40)/2 records mostly joint variation. The differential coordinate a−=(X1−X2)/2a_-=(X_1-X_2)/\sqrt2a−=(X1−X2)/2 removes SSS exactly under these hypotheses and records the discrepancy between sensors. Keeping only v+v_+v+ explains 9/(9+1)=90%9/(9+1)=90%9/(9+1)=90% of the variance; the expected squared reconstruction error per sample is 1 in the chosen units.
What disappears when compressing. The one-component reconstruction is
X^=[2020]+v+v+T(X−[2020])=[(X1+X2)/2(X1+X2)/2].\widehat X=\begin{bmatrix}20\\20\end{bmatrix}+v_+v_+^{\mathsf T}\left(X-\begin{bmatrix}20\\20\end{bmatrix}\right)=\begin{bmatrix}(X_1+X_2)/2\\(X_1+X_2)/2\end{bmatrix}.X=[2020]+v+v+T(X−[2020])=[(X1+X2)/2(X1+X2)/2].
The reconstructed sensors always agree. For the pair (22,18)(22,18)(22,18) the reconstruction is (20,20)(20,20)(20,20): the four-degree difference vanishes entirely. If the goal was to estimate a common temperature under independent noise, averaging can be reasonable. If the goal was to detect that the sensors disagree, the discarded direction is exactly the one that matters. It cannot be claimed that 90 % of the useful information was kept for both tasks (PCA, projection).
Covariance is not dynamics. The values 9 and 1 are variances, not damping rates. The vector (1,−1)(1,-1)(1,−1) looks like the disagreement pattern of two tanks, but here the matrix does not say that disagreement dies out. Predicting its evolution needs an additional temporal model (modes). The sensor model and its values were chosen for this note to separate retained variance from diagnostic relevance.