industrial//process monitoring//multivariate statistical process control//Q statistic

The Q statistic is a monitoring statistic that measures how far a new sample lies from the plane of normal operation learned by PCA, the squared length of the part of the sample the normal model cannot explain; it is used to catch faults that break the relations between variables even while every variable stays inside its own limits. A drifting sensor, a valve stuck at one opening or a leak breaks the mass and energy balances that tied the readings together, and the broken balance lands in Q.

The Q statistic is a monitoring statistic that measures how far a new sample lies from the plane of normal operation learned by PCA, the squared length of the part of the sample the normal model cannot explain; it is used to catch faults that break the relations between variables even while every variable stays inside its own limits. A drifting sensor, a valve stuck at one opening or a leak breaks the mass and energy balances that tied the readings together, and the broken balance lands in Q.

With the standardized sample xxx and the kkk principal directions VkV_kVk​ fitted on normal data, the reconstruction VkVkTxV_kV_k^{\mathsf T}xVk​VkT​x is what the normal model predicts for that sample, and Q (also called the squared prediction error, SPE) is what is left over:

Q=∥x−VkVkTx∥2=∑j=k+1p(vjTx)2.Q=\bigl\|x-V_kV_k^{\mathsf T}x\bigr\|^2=\sum_{j=k+1}^{p}\bigl(v_j^{\mathsf T}x\bigr)^2 .Q=​x−Vk​VkT​x​2=j=k+1∑p​(vjT​x)2.

The second form says the same thing in the discarded directions: Q adds up the sample's components along the directions where normal operation hardly varies.

The directions PCA throws away are the ones physics holds still.

Normal samples have almost nothing along them, so anything there is a relation breaking. A record production day moves the plant far along the normal plane with every relation intact, so the T-squared statistic rises and Q stays small; a pump whose current fell while its flow did not is close to the centre in every variable, and Q jumps.

It is a residual in the sense of fault diagnosis, with the expectation learned from data instead of written from a model: the distance between what was measured and what the normal model says should have been measured (residual as surprise traces the pattern across fields). Two redundant sensors give its smallest case: Q is their disagreement.

Its threshold is set at the 99th percentile on held-out normal data or with the Jackson and Mudholkar approximation, and an alarm is examined through the contribution of each variable to Q.

Its false alarms have known causes: curved relations that a linear plane cannot follow, an operating mode absent from the training weeks, too few components kept (normal variation then spills into Q), and slow ageing of the plant after maintenance. The same quantity reappears as the reconstruction error of an autoencoder, which is a nonlinear version of the same idea.

It is computed with T-squared on every sample in multivariate statistical process control.