mathematics//dimensionality reduction//projection
A projection is the operation that keeps the part of a vector lying in a subspace and drops the rest, and it is the point where changing coordinates turns into compressing. A **complete invertible change of basis** describes the same vector in other coordinates. A **dimensionality reduction** keeps fewer coordinates or summarizes the object with fewer parameters, and generally loses information (dimensionality reduction).
A projection is the operation that keeps the part of a vector lying in a subspace and drops the rest, and it is the point where changing coordinates turns into compressing. A complete invertible change of basis describes the same vector in other coordinates. A dimensionality reduction keeps fewer coordinates or summarizes the object with fewer parameters, and generally loses information (dimensionality reduction).
For a unit direction vvv and a centered vector x−μx-\mux−μ, the projected coordinate is a=vT(x−μ)a=v^{\mathsf T}(x-\mu)a=vT(x−μ). The scalar aaa is not the projected point in the original space; that point is x^=μ+av\hat x=\mu+avx^=μ+av.
Projecting onto a subspace works the same way with several directions. If UrU_rUr holds rrr orthonormal columns, the coordinates and the projected point are these.
a=UrT(x−μ),x^=μ+UrUrT(x−μ)a=U_r^{\mathsf T}(x-\mu),\qquad \hat x=\mu+U_rU_r^{\mathsf T}(x-\mu)a=UrT(x−μ),x^=μ+UrUrT(x−μ)
The residual x−x^x-\hat xx−x^ is perpendicular to the subspace, and that is what makes the orthogonal projection the closest point of the subspace to xxx: any other point adds a component along the subspace to an error that was already perpendicular to it. The projection matrix P=UrUrTP=U_rU_r^{\mathsf T}P=UrUrT changes nothing when applied twice (P2=PP^2=PP2=P), the algebraic signature of every projection. When the subspace is spanned by columns that are not orthonormal, as those of a regression matrix AAA, it reads P=A(ATA)−1ATP=A(A^{\mathsf T}A)^{-1}A^{\mathsf T}P=A(ATA)−1AT, and projecting the measurements onto it is least squares.
With every vector of an orthonormal basis the reconstruction is exact; keeping only some, the orthogonal part is dropped. A hyperplane is strictly an affine subspace of dimension one less than the ambient space; a reduction to arbitrary dimension uses a subspace or a low-dimensional representation, not always a hyperplane.
A criterion decides what is kept. PCA keeps the directions of largest variance and minimizes squared reconstruction error among linear subspaces of fixed dimension, for that data scale. spectral clustering builds coordinates from connectivity; it is not PCA in other notation, and its new axes are not linear combinations of the original variables (spectral coordinates). In a dynamic model, keeping the slow modes can approximate the long run, but fast modes may matter for transients, actuation or measurement; a reduction by variance does not preserve stability, conservation, controllability or observability by itself (controllability and observability).
An engineering example shows how to avoid a bad reduction. If two sensors measure a common temperature, the sum mode can explain almost all the variance while the difference mode, though small, identifies a calibration fault. Discarding it can improve compression and worsen diagnosis (redundant sensors). The decision to reduce must state its goal: reconstruct signals, detect faults, simulate the long run, design control or group cases. No single notion of relevant information is valid for all of them.
In process monitoring the projection is the model of normal operation. A PCA fitted on weeks of healthy data keeps kkk directions VkV_kVk, and VkVkTxV_kV_k^{\mathsf T}xVkVkTx projects each new standardized sample onto that plane of normal behaviour. The distance from the sample to its projection, the Q statistic, grows when the physical relations between the variables break (a drifting sensor, a sticking valve) even while every variable stays inside its own limits (multivariate statistical process control).