Python KNN, short for K-Nearest Neighbors, is the simplest machine learning algorithm to reason about: it predicts by finding the most similar training examples and copying their answer. We cover distance metrics, how to pick K, the curse of dimensionality, and where KNN shines in real projects.

“The nearest neighbor rule achieves an error rate no worse than twice the Bayes error rate.”

Thomas Cover, Nearest Neighbor Pattern Classification

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0 | Difficulty: Intermediate | Reading Time: 14 minutes

Python KNN is the friendliest machine learning algorithm to wrap your head around. Picture moving to a new street and wondering whether it is a quiet area or a noisy one. You glance at the five houses closest to you. If four are calm families and one throws parties, you bet the street is calm. That is the whole idea. To label a new data point, KNN looks at the K closest training examples and takes a majority vote.

To predict a number, it averages the values of those K neighbors. There is no real training step. KNN just memorizes every training example and does all the thinking at prediction time, so training feels instant but predictions get slow once the dataset is huge.

Only one setting really matters, and that is K, the number of neighbors you ask. K=1 means “copy whatever the single closest point says”. That is jumpy and noisy, like trusting the opinion of one random neighbor. A very large K like 100 averages a huge crowd, which smooths everything out so much that you lose the local detail. The sweet spot sits in between. For classification, an odd number somewhere from 3 to 15 usually works well (odd so a tie vote cannot happen).

KNN is a great fit for recommendation systems (“people who bought this also bought…”), anomaly detection (“this point has no close neighbors, so it looks suspicious”), and any problem where being similar matters more than following a learned rule. Its weak spot is high-dimensional data. When you have 100 or more features, the curse of dimensionality kicks in and every point ends up roughly the same distance from every other point, so “nearest” stops meaning much.

Prerequisites

How Python KNN Works: No Training, All Prediction

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

Read the diagram top to bottom and you have the entire Python KNN algorithm. A new point arrives with an unknown class. KNN measures its distance to every training point, sorts those distances, and keeps the K closest. Here K=3, and the three nearest neighbors are 2 Blue and 1 Red, so the majority vote says Blue. The boxes on the right show what K does to the result: K=1 copies the single closest point and gets noisy, a mid-range K like 5 stays balanced, and a giant K like 50 smooths so hard that it underfits.

Notice there is no training box anywhere. All the work happens at prediction time, which is exactly why KNN feels instant to fit but turns slow once you have to scan a large pile of training points for every single prediction.

📄 knn_basics.py: KNN classification step by step

import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_iris

# Iris dataset: the classic KNN example
iris = load_iris()
X_scaled = StandardScaler().fit_transform(iris.data)

# Effect of K
print(f"{'K':>4} | {'CV Score':>10} | {'Assessment'}")
print("-" * 35)
for k in [1, 3, 5, 7, 9, 15, 25, 50]:
    knn = KNeighborsClassifier(n_neighbors=k)
    scores = cross_val_score(knn, X_scaled, iris.target, cv=5)
    assessment = "Overfitting" if k <= 1 else "Underfitting" if k >= 30 else "Good"
    print(f"{k:>4} | {scores.mean():>9.3f} | {assessment}")

▶ Output

   K |   CV Score | Assessment
-----------------------------------
   1 |     0.947 | Overfitting
   3 |     0.953 | Good
   5 |     0.960 | Good
   7 |     0.953 | Good
   9 |     0.960 | Good
  15 |     0.947 | Good
  25 |     0.940 | Good
  50 |     0.873 | Underfitting

What happened here: K=1 (0.947) sits below the mid-range peak because it copies whatever single point is nearest, so one stray outlier can flip the answer. Accuracy climbs and peaks around K=5 to K=9 (0.960), then slowly slides back down as the neighborhood grows. By K=50 it has fallen to 0.873, since the vote now drags in points from other flower species and drowns out the local signal. The numbers are cross-validation scores, so yours may shift by a thousandth or two, but the shape stays the same: small K too jumpy, large K too blurry, a middle K just right.

Use an odd K for classification so a vote can never tie, and K=5 to K=7 is a solid place to start.

Distance Metrics: How “Close” Is Close?

KNN lives or dies on how it measures “close”, and there is more than one honest way to do that. Think about getting from one corner of a city block to another. Euclidean distance is the straight line a bird would fly. Manhattan distance is the path a taxi takes, only along the streets, so it adds up the horizontal blocks plus the vertical blocks. Minkowski is the general formula that both of those are special cases of (p=2 gives Euclidean, p=1 gives Manhattan). Chebyshev is the king-on-a-chessboard move: the distance is just the single biggest step on any one axis. Most of the time Euclidean is the right default, but it is worth knowing the others exist.

📄 distance_metrics.py: Euclidean vs Manhattan vs Minkowski

import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris

iris = load_iris()
X_scaled = StandardScaler().fit_transform(iris.data)

# Compare distance metrics
metrics = {
    "Euclidean (p=2)": {"metric": "euclidean"},
    "Manhattan (p=1)": {"metric": "manhattan"},
    "Minkowski (p=3)": {"metric": "minkowski", "p": 3},
    "Chebyshev (p=inf)": {"metric": "chebyshev"},
}

print(f"{'Metric':<20} | {'CV Score':>10}")
print("-" * 34)
for name, params in metrics.items():
    knn = KNeighborsClassifier(n_neighbors=5, **params)
    scores = cross_val_score(knn, X_scaled, iris.target, cv=5)
    print(f"{name:<20} | {scores.mean():>9.3f}")

# Manual distance calculation
p1 = np.array([1, 2])
p2 = np.array([4, 6])
euclidean = np.sqrt(np.sum((p1 - p2) ** 2))
manhattan = np.sum(np.abs(p1 - p2))
print(f"\nPoints: {p1} and {p2}")
print(f"Euclidean distance: {euclidean:.2f} (straight line)")
print(f"Manhattan distance: {manhattan:.0f} (grid/taxi distance)")

▶ Output

Metric               |   CV Score
----------------------------------
Euclidean (p=2)      |     0.960
Manhattan (p=1)      |     0.947
Minkowski (p=3)      |     0.953
Chebyshev (p=inf)    |     0.933

Points: [1 2] and [4 6]
Euclidean distance: 5.00 (straight line)
Manhattan distance: 7 (grid/taxi distance)

What happened here: On the Iris data all four metrics land in the same ballpark, with Euclidean just ahead at 0.960. That is normal. For most everyday problems the choice of distance metric nudges the score by a hair, not a mile, so reach for the default (Euclidean) first and only switch if a test set tells you to. The hand calculation at the bottom makes the difference concrete: from (1, 2) to (4, 6) the straight-line Euclidean distance is 5.00, while the taxi-route Manhattan distance is 7, because the taxi has to cover 3 blocks across plus 4 blocks up instead of cutting the corner.

KNN for Regression

Python KNN is not only for picking a category. It can predict a number too, and the change is tiny. Instead of taking a majority vote among the K neighbors, it averages their values. Want to estimate a flat’s price? Find the K most similar flats and average what they sold for. That is literally how a property agent eyeballs a price: “places like this one nearby went for around so much”. One nice upgrade is distance weighting. With weights="distance", neighbors that sit closer get a louder vote than ones further away, which usually sharpens the estimate.

📄 knn_regression.py: predicting house prices

import numpy as np
from sklearn.neighbors import KNeighborsRegressor
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import cross_val_score

rng = np.random.default_rng(42)

# House price data
n = 300
X = np.column_stack([
    rng.integers(500, 3000, n),   # area
    rng.integers(1, 5, n),         # bedrooms
    rng.uniform(1, 25, n),         # distance to metro
])
y = 50 * X[:, 0] + 200000 * X[:, 1] - 8000 * X[:, 2] + rng.normal(0, 30000, n)

X_scaled = StandardScaler().fit_transform(X)

for k in [3, 5, 10, 20]:
    knn = KNeighborsRegressor(n_neighbors=k, weights="distance")
    scores = cross_val_score(knn, X_scaled, y, cv=5, scoring="r2")
    print(f"K={k:>2}: R² = {scores.mean():.3f}")

# Uniform voting (every neighbor counts equally)
# vs distance voting (closer neighbors count more)
for weights in ["uniform", "distance"]:
    knn = KNeighborsRegressor(n_neighbors=5, weights=weights)
    scores = cross_val_score(knn, X_scaled, y, cv=5, scoring="r2")
    print(f"\nWeights={weights}: R² = {scores.mean():.3f}")

▶ Output

K= 3: R² = 0.976
K= 5: R² = 0.975
K=10: R² = 0.972
K=20: R² = 0.963

Weights=uniform: R² = 0.972

Weights=distance: R² = 0.975

What happened here: An R² of 0.97 means KNN explains about 97 percent of the variation in our (made up) house prices, which is strong. A small K like 3 edges ahead here, and the score eases off as K grows toward 20 and the average gets blurrier. Distance weighting (0.975) beats plain uniform voting (0.972) by a whisker, which is the usual pattern: letting closer neighbors speak louder helps a little. The data is synthetic and seeded with default_rng(42), so you will get these exact numbers if you run it. On real data the values move, but the trend holds.

The Curse of Dimensionality

This is the one weakness that bites KNN the hardest, so it is worth a clear picture. Imagine looking for your closest friend in a room. Easy. Now spread everyone across a whole city, then across a country, then across a continent. The further apart you let people drift, the more “nearest” loses its meaning, because almost everyone ends up about equally far away. Adding features to your data does the same thing. In very high dimensions the distance between the closest pair and the farthest pair shrinks until they are nearly identical, and once “near” and “far” look the same, KNN has nothing left to vote on.

📄 curse_of_dimensionality.py: why KNN fails in high dimensions

from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
from sklearn.datasets import make_classification

# Same classification task, but with more and more features.
# Only 5 features ever carry real signal. Every extra one is noise.
print(f"{'Features':>8} | {'KNN Score':>10} | {'Note'}")
print("-" * 45)
for n_features in [5, 10, 25, 50, 100, 500]:
    X, y = make_classification(n_samples=500, n_features=n_features,
                               n_informative=5, n_redundant=0,
                               n_repeated=0, random_state=42)
    knn = KNeighborsClassifier(n_neighbors=5)
    scores = cross_val_score(knn, X, y, cv=5)
    note = "Good" if scores.mean() > 0.85 else "Degraded" if scores.mean() > 0.7 else "Failed"
    print(f"{n_features:>8} | {scores.mean():>9.3f} | {note}")

print(f"\nAs dimensions increase, all points become equally distant.")
print(f"With 500 features and only 5 informative, KNN cannot find the signal.")
print(f"Solutions: use PCA or feature selection before KNN.")

▶ Output

Features |  KNN Score | Note
---------------------------------------------
       5 |     0.926 | Good
      10 |     0.918 | Good
      25 |     0.840 | Degraded
      50 |     0.778 | Degraded
     100 |     0.722 | Degraded
     500 |     0.620 | Failed

As dimensions increase, all points become equally distant.
With 500 features and only 5 informative, KNN cannot find the signal.
Solutions: use PCA or feature selection before KNN.

What happened here: The signal in this data never changes. There are always exactly 5 features that actually matter. All we do is pad the data with more and more pure-noise columns. Watch the score fall off a cliff: a clean 0.926 at 5 features, still fine at 10, but sliding into “Degraded” by 25 features and crashing to 0.620 (“Failed”) at 500. The 5 useful features are still in there, but they are buried so deep in junk that the distance calculation can no longer tell a real neighbor from a random one.

The fix is to cut the dimensions before KNN ever sees the data: run Principal Component Analysis (PCA) or pick the features that matter, then let KNN work in that smaller, cleaner space.

Common Mistakes

The single most common Python KNN bug is forgetting to scale your features, and it is sneaky because the code still runs and gives an answer. The answer is just quietly wrong. KNN adds up differences across every feature, so a column with big raw numbers steamrolls the rest. Picture comparing people by salary (say 30,000 to 150,000 rupees) and age (20 to 60). A two-year age gap is a rounding error next to a salary gap of thousands, so the distance is basically “whoever earns a similar amount”, and age stops counting at all. Scaling puts every feature on the same footing first, so each one gets a fair say.

❌ Mistake: Forgetting to scale features

# KNN uses distances. If salary is in [30000, 150000] and age is in [20, 60],
# salary dominates every distance calculation.
# A 1000 rupee salary gap counts for more than a 10-year age gap.
# ALWAYS scale features before KNN.
print("KNN is distance-based. Unscaled features make large-range")
print("features dominate. Use StandardScaler or MinMaxScaler.")

▶ Output

KNN is distance-based. Unscaled features make large-range
features dominate. Use StandardScaler or MinMaxScaler.

Practice Exercises

1. Exercise 1: Train KNN with k=5, evaluate accuracy.
2. Exercise 2: Plot accuracy vs k (1-50) for optimal k.
3. Exercise 3: Implement KNN from scratch with distance weighting. Compare sklearn.

Conclusion

You now have the whole picture of Python KNN. It predicts by finding the K most similar training examples and either taking a majority vote (classification) or averaging their values (regression). You saw how K controls the balance between jumpy (small K) and blurry (large K), why an odd K avoids ties, how distance metrics like Euclidean and Manhattan measure “close”, and why distance weighting can sharpen predictions. Two things matter most in practice: always scale your features first, and watch out for the curse of dimensionality, which quietly wrecks KNN once you pile on too many features. When that happens, cut dimensions with PCA or feature selection before KNN ever sees the data.

Next up is Naive Bayes, a probabilistic classifier that, unlike KNN, actually learns from the data during training and stays fast even on high-dimensional text. For the full path from Python basics to machine learning, browse the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on KNN

These come from real screens and onsites. Practice answering before you read each answer.

Q: Your KNN model trained in milliseconds, but production predictions are timing out. Why does that happen by design, and what are your options?

KNN does no real work during training. It just stores the entire training set. All the computation happens at prediction time, when it measures the distance from the new point to every stored point. The trade-off is a fast fit but slow predictions: each prediction is roughly O(n*d) for n samples and d features. scikit-learn softens this with KD-trees and Ball-trees, which it picks automatically for lower-dimensional data.

Q: Why must you scale features before using KNN?

KNN sums differences across all features to compute distance, so any feature with a large numeric range dominates. If salary ranges over tens of thousands and age over a few dozen, the distance becomes “whoever earns a similar amount” and age is effectively ignored. StandardScaler or MinMaxScaler puts every feature on the same footing so each contributes fairly. Skipping this step still runs without error, which is what makes the bug so sneaky.

Q: How do you choose K, and why prefer an odd value for classification?

Use cross-validation across a range of K values (a common span is 1 to sqrt(n)) and pick the K where validation accuracy peaks. Small K overfits and is noisy; large K underfits and blurs local detail, so the best value sits in the middle, often 5 to 9. An odd K is preferred for binary classification because it prevents a tied vote between two classes.

Q: What is the difference between uniform and distance weighting?

With weights="uniform", every one of the K neighbors gets an equal vote. With weights="distance", closer neighbors count more, since their vote is weighted by the inverse of their distance. Distance weighting usually gives a small accuracy bump and is especially helpful for regression, where a nearby example is a better price estimate than a far one. It also reduces the impact of choosing a slightly-too-large K.

Q: A teammate says KNN works great on their 3-feature demo but scores near random on a 400-feature text dataset. What do you check first?

This is the curse of dimensionality. With hundreds of features, the distance between the nearest and farthest points collapses toward being equal, so “nearest” stops carrying signal and KNN degrades to guessing. First confirm how many features actually carry information, then reduce dimensions with PCA or feature selection before KNN, or switch to a model that handles high-dimensional data well, such as Naive Bayes or logistic regression. Also verify the features were scaled.

Q: Your KNN model is accurate in a notebook but far too slow to serve predictions in production. What options do you have?

Prediction cost grows with the training set size because each query scans many stored points. Options: enable an approximate or tree-based index (KD-tree or Ball-tree via the algorithm parameter) for lower dimensions, reduce dimensionality so those trees stay effective, downsample or prototype-select the training data to shrink n, or precompute and cache predictions for common queries. If latency stays unacceptable, a model that does its work at training time (like logistic regression or a tree ensemble) is often the better fit for serving.

Series: Python + AI/ML Cookbook, Part 5: Machine Learning

Further reading: scikit-learn documentation is the authoritative source on this.

Related Posts

Previous: SVM in Python: Support Vector Machines, Kernels and Margins

Next: ML: Naive Bayes, Probabilistic Classification

Series Home: Python + AI/ML Tutorial Series