ML//unsupervised learning//PCA//redundant sensors
Two redundant sensors make the worked example of a low-variance direction that is the most important one for diagnosis. Both measure a common temperature.
Two redundant sensors make the worked example of a low-variance direction that is the most important one for diagnosis. Both measure a common temperature.
X1=20+S+η1,X2=20+S+η2X_1=20+S+\eta_1,\qquad X_2=20+S+\eta_2X1=20+S+η1,X2=20+S+η2
SSS is process variation with zero mean and variance 4. The noises η1,η2\eta_1,\eta_2η1,η2 have zero mean and variance 1, independent of each other and of SSS. Temperatures are in degrees Celsius, variances in degrees squared.
The pairs share a signal. Each sensor has variance 4+1=54+1=54+1=5, and their covariance is 4 through the shared signal, which gives this covariance matrix and correlation.
Σ=[5445],ρ12=45=0.8\Sigma=\begin{bmatrix}5&4\\4&5\end{bmatrix},\qquad \rho_{12}=\tfrac45=0.8Σ=[5445],ρ12=54=0.8
The covariance has units, the correlation does not. The positive correlation expresses joint variation under the model; it does not by itself prove the shared cause the model assumes (covariance).
Two directions carry meaning. v+=(1,1)T/2\ca{v_+}=(1,1)^{\mathsf T}/\sqrt2v+=(1,1)T/2 with λ+=9\ca{\lambda_+}=9λ+=9, and v−=(1,−1)T/2\cb{v_-}=(1,-1)^{\mathsf T}/\sqrt2v−=(1,−1)T/2 with λ−=1\cb{\lambda_-}=1λ−=1 (covariance matrix). The common mode in green, the difference in blue. The common coordinate a+=(X1+X2−40)/2\ca{a_+}=(X_1+X_2-40)/\sqrt2a+=(X1+X2−40)/2 records mostly joint variation. The differential coordinate a−=(X1−X2)/2\cb{a_-}=(X_1-X_2)/\sqrt2a−=(X1−X2)/2 removes SSS exactly under these hypotheses and records the discrepancy between sensors. Keeping only v+\ca{v_+}v+ explains 9/(9+1)=90%9/(9+1)=90%9/(9+1)=90% of the variance; the expected squared reconstruction error per sample is 1 in the chosen units.
Compressing makes the disagreement disappear. The one-component reconstruction replaces both readings by their average.
X^=[2020]+v+v+T(X−[2020])=[(X1+X2)/2(X1+X2)/2]\widehat X=\begin{bmatrix}20\\20\end{bmatrix}+\ca{v_+}\ca{v_+}^{\mathsf T}\left(X-\begin{bmatrix}20\\20\end{bmatrix}\right)=\begin{bmatrix}(X_1+X_2)/2\\(X_1+X_2)/2\end{bmatrix}X=[2020]+v+v+T(X−[2020])=[(X1+X2)/2(X1+X2)/2]
The reconstructed sensors always agree. For the pair (22,18)(22,18)(22,18) the reconstruction is (20,20)(20,20)(20,20): the four-degree difference vanishes entirely. If the goal was to estimate a common temperature under independent noise, averaging can be reasonable. If the goal was to detect that the sensors disagree, the discarded direction is exactly the one that matters. It cannot be claimed that 90 % of the useful information was kept for both tasks (PCA, projection).
A plant is this example at scale, because physics ties its variables. In a compressor station, temperatures, pressures, flows and motor currents move together through mass and energy balances, so forty sensors usually have far fewer real degrees of freedom, and a handful of components explain almost all normal variation. The discarded directions hold noise and, more usefully, whatever should not happen: a drifting sensor or a leak breaks a balance and shows up there even with every reading inside its limits. Watching that residual is the Q statistic.
Covariance is not dynamics. The values 9 and 1 are variances, not damping rates. The vector (1,−1)(1,-1)(1,−1) looks like the disagreement pattern of two tanks, but here the matrix does not say that disagreement dies out. Predicting its evolution needs an additional temporal model (modes). The sensor model and its values were chosen for this note to separate retained variance from diagnostic relevance.