mathematics//dimensionality reduction//projection

When changing coordinates keeps the object and when it compresses it. A **complete invertible change of basis** describes the same vector in other coordinates. A **dimensionality reduction** keeps fewer coordinates or summarizes the object with fewer parameters, and generally loses information (dimensionality reduction).

When changing coordinates keeps the object and when it compresses it. A complete invertible change of basis describes the same vector in other coordinates. A dimensionality reduction keeps fewer coordinates or summarizes the object with fewer parameters, and generally loses information (dimensionality reduction).

For a unit direction vvv and a centered vector x−μx-\mux−μ, the projected coordinate is a=vT(x−μ)a=v^{\mathsf T}(x-\mu)a=vT(x−μ). The scalar aaa is not the projected point in the original space; that point is x^=μ+av\hat x=\mu+avx^=μ+av.

Projecting onto a subspace. If UrU_rUr​ holds rrr orthonormal columns,

a=UrT(x−μ),x^=μ+UrUrT(x−μ).a=U_r^{\mathsf T}(x-\mu),\qquad \hat x=\mu+U_rU_r^{\mathsf T}(x-\mu).a=UrT​(x−μ),x^=μ+Ur​UrT​(x−μ).

The residual x−x^x-\hat xx−x^ is perpendicular to the subspace. With every vector of an orthonormal basis the reconstruction is exact; keeping only some, the orthogonal part is dropped. "Hyperplane" strictly means an affine subspace of dimension one less than the ambient space; a reduction to arbitrary dimension uses a subspace or a low-dimensional representation, not always a hyperplane.

Which criterion decides what is kept. PCA keeps the directions of largest variance and minimizes squared reconstruction error among linear subspaces of fixed dimension, for that data scale. spectral clustering builds coordinates from connectivity; it is not PCA in other notation, and its new axes are not linear combinations of the original variables (spectral coordinates). In a dynamic model, keeping the slow modes can approximate the long run, but fast modes may matter for transients, actuation or measurement; a reduction by variance does not preserve stability, conservation, controllability or observability by itself (controllability and observability).

The engineering example that avoids a bad reduction. If two sensors measure a common temperature, the sum mode can explain almost all the variance while the difference mode, though small, identifies a calibration fault. Discarding it can improve compression and worsen diagnosis (redundant sensors). The decision to reduce must state its goal: reconstruct signals, detect faults, simulate the long run, design control or group cases. There is no single notion of "relevant information" valid for all of them.