K-nearest neighbors algorithm

lastmod 2025-01-06created 2019-12-064 edits

The k-nearest neighbors algorithm (k-NN) is a method that classifies an observation based on the closest observations among the training samples, using the observations' features. It is regarded as one of the simplest methods in machine learning.

Algorithm

kNN works in the following order.

1. Iterate over every measurement in the dataset:
   1. Compute the distance X between inX and the current measurement.
   2. Sort the distances X in ascending order.
   3. Take the k items with the shortest distance to inX.
   4. Find the most common class among the k items.
   5. Return the most common class as the predicted class of inX.

See also

• Machine learning
• Hyperparameter
• Overfitting
• Backpropagation
• Norm
• Nearest neighbor search

Favorite site

• Wikipedia (en) description of the K-NN algorithm
• [Recommended] Pattern KNN Clustering
• Introduction to machine learning models (3) - KNN (K-Nearest Neighbor)