mathematics//linear algebra//matrix factorization//eigendecomposition
The eigendecomposition is the factorization of a square matrix into its eigenvectors and its eigenvalues, \(A=V\Lambda V^{-1}\), and it is how a coupled linear system is split into independent scalar problems, one per mode: a vibrating structure into its mode shapes, a thermal network into its time constants, a Markov chain into its long-run behaviour and its forgetting rate.
The eigendecomposition is the factorization of a square matrix into its eigenvectors and its eigenvalues, A=VΛV−1A=V\Lambda V^{-1}A=VΛV−1, and it is how a coupled linear system is split into independent scalar problems, one per mode: a vibrating structure into its mode shapes, a thermal network into its time constants, a Markov chain into its long-run behaviour and its forgetting rate.
A=VΛV−1,xk=Akx0=∑i=1nci λik viA=V\Lambda V^{-1},\qquad x_k=A^kx_0=\sum_{i=1}^{n}c_i\,\lambda_i^{k}\,v_iA=VΛV−1,xk=Akx0=i=1∑nciλikvi
VVV holds the eigenvectors in its columns, Λ\LambdaΛ the eigenvalues on its diagonal and cic_ici is how much of mode iii the initial condition contained. In the basis of the eigenvectors the matrix is diagonal, so each mode lives its own life: with ∣λi∣<1|\lambda_i|<1∣λi∣<1 it fades, with ∣λi∣>1|\lambda_i|>1∣λi∣>1 it grows. Two zones of a workshop coupled thermally, with temperatures above outside evolving each minute by A=[0.950.030.030.95]A=\begin{bmatrix}0.95&0.03\0.03&0.95\end{bmatrix}A=[0.950.030.030.95], have eigenvectors (1,1)(1,1)(1,1) and (1,−1)(1,-1)(1,−1) with eigenvalues 0.98 and 0.92. The mean temperature relaxes slowly, with a time constant of about fifty minutes, and the difference between zones disappears in about twelve: a common mode and a differential mode, each with its own clock.
With nnn independent eigenvectors, one hard coupled problem becomes nnn easy scalar ones. Powers, exponentials and stability of AAA all reduce to the same operations on numbers: Ak=VΛkV−1A^k=V\Lambda^kV^{-1}Ak=VΛkV−1 and eAt=VeΛtV−1e^{At}=Ve^{\Lambda t}V^{-1}eAt=VeΛtV−1 (matrix exponential).
Not every matrix has one. A matrix with too few independent eigenvectors (a defective matrix, such as a double integrator) cannot be diagonalized, and its response contains terms like t eλtt,e^{\lambda t}teλt that no single mode describes. Matrices close to that case have nearly parallel eigenvectors, an ill-conditioned VVV and transients the eigenvalues hide (non-normal matrix).
Symmetric matrices are the pleasant case: real eigenvalues and an orthonormal VVV, so A=VΛVTA=V\Lambda V^{\mathsf T}A=VΛVT. Covariances, stiffness matrices and graph Laplacians fall here, and the symmetric solver (np.linalg.eigh) is faster than the general eig and guaranteed to return real results.
For a symmetric positive semidefinite matrix the eigendecomposition and the SVD coincide; for anything else they answer different questions, the first about what repeated application does, the second about how much one application can stretch.