K-Nearest Neighbors
KNN classification/regression finds the K nearest training samples to a new sample, voting for classification and averaging for regression, with no training phase.
Purpose
To provide a simple non-parametric lazy-learning algorithm suited to low dimensions and small datasets.
Mechanism
Find the K nearest neighbors of a query point (under Euclidean, Manhattan, or other distances); classification takes a majority vote, regression takes the mean or a weighted mean. A distance metric and the choice of K are required. KD-trees accelerate low-dimensional queries.
Use cases
- Classification and regression
- Recommender systems
- Pattern recognition