Regularization
Regularization adds a parameter penalty (L1/L2) to the loss function to limit model complexity and prevent overfitting.
Purpose
To control model complexity and improve generalization.
Mechanism
L2 (Ridge): λ||w||^2 shrinks parameters toward zero without sparsity. L1 (Lasso): λ||w||_1 induces sparse parameters (feature selection). The elastic net αL1+(1-α)L2 combines both. λ is chosen by cross-validation.
Use cases
- Overfitting defense
- Feature selection
- Model generalization