Singular Value Decomposition
Singular value decomposition factors any matrix A as UΣV^T, with U and V orthogonal and Σ diagonal (the singular values); it is a core tool of matrix analysis.
Purpose
To underpin low-rank approximation, PCA, pseudo-inverses, and condition numbers.
Mechanism
A=UΣV^T, with singular values σ_i=√(eigenvalues of A^T A). Truncated SVD takes the top k singular values for the best rank-k approximation (Eckart-Young). It is numerically stable but costs O(min(mn^2, m^2n)).
Use cases
- PCA dimensionality reduction
- Recommender systems
- Matrix pseudo-inverse