Logistic Regression
Logistic regression maps a linear combination to a probability via the sigmoid, P(y=1|x)=σ(w·x+b), trained with cross-entropy loss for binary classification.
Purpose
To provide a baseline binary classifier that outputs probabilities and remains interpretable.
Mechanism
σ(z)=1/(1+e^(-z)). The loss L=-∑[y log p+(1-y) log(1-p)] is convex. It is optimized by gradient descent or Newton's method (IRLS). Multiclass uses softmax. L1/L2 regularization prevents overfitting.
Use cases
- Binary classification
- Probability prediction
- Interpretable models