Principal Component Analysis
PCA finds the maximum-variance directions via eigendecomposition of the covariance matrix, projecting high-dimensional data onto a low-dimensional subspace to reduce dimensionality.
Purpose
To provide the most widely used linear dimensionality reduction, decorrelation, and feature extraction.
Mechanism
Center the data, compute the covariance matrix C=X^T X/(n-1), and take the eigenvectors of the top k eigenvalues as the projection matrix. This is equivalent to truncated SVD and retains the maximum-variance directions.
Use cases
- Data dimensionality reduction
- Feature extraction
- Visualization