Random Forest
A random forest trains many decision trees, each on a bootstrap sample with a random feature subset, and votes/averages at prediction time, reducing variance.
Purpose
To provide a robust, easy-to-use ensemble that reduces decision-tree overfitting.
Mechanism
Each tree trains on a bootstrap sample, and each split considers only a random feature subset (√p). Prediction takes a majority vote (classification) or the mean (regression). Out-of-bag error estimates the generalization error.
Use cases
- Classification and regression
- Feature importance
- Robust prediction