Expectation-Maximization
The EM algorithm alternates an E step (computing expectations of latent variables) with an M step (maximizing the expected log-likelihood) to fit models with latent variables.
Purpose
To obtain maximum-likelihood estimates for probabilistic models with latent variables, used in GMM and HMM training.
Mechanism
E step: Q(θ|θ_t)=E[log p(X,Z|θ)|X,θ_t]. M step: θ_{t+1}=argmax Q. Each iteration never decreases the likelihood, converging to a local optimum. Used for GMM clustering and HMM training.
Use cases
- GMM training
- HMM training
- Missing-data inference