K-means clustering in Python is how you find hidden groups in data that has no labels. You hand the algorithm a pile of raw points, tell it how many groups to look for, and it sorts everything into clusters on its own. This post walks through how k-means works step by step, the elbow method for picking the number of clusters, silhouette analysis for judging cluster quality, and a real customer segmentation example with scikit-learn.

“Clustering is the most important unsupervised learning problem. It deals with finding a structure in a collection of unlabeled data.”

Andrew Ng, Stanford CS229

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0 | Difficulty: Intermediate | Reading Time: 18 minutes

Think about how you sort a basket of clean laundry. Nobody hands you labels. You just notice that socks go with socks, shirts go with shirts, and towels make their own pile. You group by similarity. That is exactly what k-means does, only with numbers instead of clothes. Every supervised algorithm in this series so far needed labels: someone had to tag an email “spam” or write down that a house sold for $350,000 before the model could learn. K-means throws that requirement away. You give it unlabeled data and a number K, and it discovers the groups for you.

The algorithm is short enough to say in one breath: pick K random centers, send every point to its nearest center, slide each center to the middle of the points that joined it, then repeat until nothing moves. Four steps and the clusters fall out. The genuinely hard part is not the algorithm. It is choosing a sensible K and knowing whether the groups you got mean something or are just noise dressed up as structure.

K-means is fast (its cost grows linearly with the number of points), it scales to large datasets, and it is the sensible first thing to reach for. Its main weakness is that it quietly assumes your clusters are roughly round and roughly the same size. When that assumption breaks, you switch to hierarchical clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise), which is the next post.

Prerequisites

How K-Means Works, Step by Step

Before any code, look at the shape of the algorithm. The diagram below is the whole thing. Steps 3 and 4 form a loop: assign points, move centroids, assign again, move again. Each pass nudges the centers closer to the true heart of each group. When a full pass leaves the centers where they already were, the algorithm has converged and stops.

The loop between steps 3 and 4 is the engine of k-means clustering. Each pass reassigns points and shifts centroids. When the centroids stop moving (or move less than a tiny tolerance), the algorithm has converged. Scikit-learn allows up to 300 iterations by default, but on clean data most runs finish in well under 20.

Here is the same algorithm in plain NumPy, with no scikit-learn, so you can see every moving part. Read it top to bottom: initialize centroids, loop, assign, recompute, check for convergence.

📄 kmeans_from_scratch.py: K-Means in plain NumPy

import numpy as np

def kmeans(X, k, max_iters=100, tol=1e-4):
    """K-Means clustering from scratch."""
    rng = np.random.default_rng(42)
    # Step 1: Initialize centroids randomly from data points
    centroids = X[rng.choice(len(X), k, replace=False)]

    for iteration in range(max_iters):
        # Step 2: Assign each point to nearest centroid
        distances = np.linalg.norm(X[:, np.newaxis] - centroids, axis=2)
        labels = np.argmin(distances, axis=1)

        # Step 3: Recompute centroids
        new_centroids = np.array([X[labels == i].mean(axis=0) for i in range(k)])

        # Check convergence
        shift = np.linalg.norm(new_centroids - centroids)
        if shift < tol:
            print(f"  Converged after {iteration + 1} iterations (shift={shift:.6f})")
            break
        centroids = new_centroids

    return labels, centroids

# Generate 3 clusters of 2D points
rng = np.random.default_rng(42)
cluster_1 = rng.normal(loc=[2, 2], scale=0.5, size=(50, 2))
cluster_2 = rng.normal(loc=[7, 7], scale=0.5, size=(50, 2))
cluster_3 = rng.normal(loc=[2, 8], scale=0.5, size=(50, 2))
X = np.vstack([cluster_1, cluster_2, cluster_3])

labels, centroids = kmeans(X, k=3)

print(f"Cluster sizes: {np.bincount(labels)}")
print(f"Centroids:")
for i, c in enumerate(centroids):
    print(f"  Cluster {i}: ({c[0]:.2f}, {c[1]:.2f})")

▶ Output

  Converged after 2 iterations (shift=0.000000)
Cluster sizes: [50 50 50]
Centroids:
  Cluster 0: (7.00, 6.99)
  Cluster 1: (1.98, 1.97)
  Cluster 2: (1.99, 7.95)

What happened here: The algorithm locked onto all three groups in just two iterations. We built the data around the centers (2, 2), (7, 7), and (2, 8), and the three recovered centroids land almost exactly on those spots: (7.00, 6.99), (1.98, 1.97), and (1.99, 7.95). One thing to notice that trips up beginners: the cluster numbers are not meaningful. Cluster 0 here is the (7, 7) group, not the (2, 2) group, purely because of where the random centroids started. K-means gives you groupings, never a fixed order. Convergence was this fast because the clusters barely overlap. Real data is messier: groups bleed into each other, sizes differ, and the right K is not handed to you.

One Iteration by Hand

The NumPy version hides the arithmetic behind vectorized operations. Let us strip it down to numbers you can check in your head. Picture six points sitting on a number line: 1, 2, 3, then a gap, then 10, 11, 12. Two groups are obvious to your eye. We will hand k-means two starting centroids placed badly on purpose, at 2 and 9, and watch one assignment step and one update step fix them.

📄 kmeans_by_hand.py: one assign-and-update pass on six points

import numpy as np

# Six points on a line, two obvious groups: low (1,2,3) and high (10,11,12)
points = np.array([1.0, 2.0, 3.0, 10.0, 11.0, 12.0])

# Start with two centroids placed badly on purpose
c0, c1 = 2.0, 9.0
print(f"Start: centroid_0 = {c0}, centroid_1 = {c1}")

# --- One assignment step: each point joins the nearer centroid ---
assign = ["c0" if abs(p - c0) <= abs(p - c1) else "c1" for p in points]
print("Distances and assignment:")
for p, a in zip(points, assign):
    print(f"  point {p:>4}: |{p}-{c0}|={abs(p-c0):.0f}, |{p}-{c1}|={abs(p-c1):.0f} -> {a}")

# --- One update step: each centroid moves to the mean of its members ---
group0 = points[[a == "c0" for a in assign]]
group1 = points[[a == "c1" for a in assign]]
new_c0 = group0.mean()
new_c1 = group1.mean()
print(f"New centroid_0 = mean({[float(x) for x in group0]}) = {new_c0}")
print(f"New centroid_1 = mean({[float(x) for x in group1]}) = {new_c1}")

▶ Output

Start: centroid_0 = 2.0, centroid_1 = 9.0
Distances and assignment:
  point  1.0: |1.0-2.0|=1, |1.0-9.0|=8 -> c0
  point  2.0: |2.0-2.0|=0, |2.0-9.0|=7 -> c0
  point  3.0: |3.0-2.0|=1, |3.0-9.0|=6 -> c0
  point 10.0: |10.0-2.0|=8, |10.0-9.0|=1 -> c1
  point 11.0: |11.0-2.0|=9, |11.0-9.0|=2 -> c1
  point 12.0: |12.0-2.0|=10, |12.0-9.0|=3 -> c1
New centroid_0 = mean([1.0, 2.0, 3.0]) = 2.0
New centroid_1 = mean([10.0, 11.0, 12.0]) = 11.0

What happened here: The assignment step is just a distance comparison. Point 3 is one unit from centroid_0 and six units from centroid_1, so it joins c0. Every low point picks c0, every high point picks c1. The update step then slides each centroid to the average of its members. Centroid_1 was sitting at 9, but the mean of its points (10, 11, 12) is 11, so it shifts to 11 where it belongs. Run another pass and nothing changes, so the algorithm has converged. That distance check plus mean update, repeated, is the entire algorithm. Everything scikit-learn does on top of this is speed and smarter starting positions.

Choosing K with the Elbow Method

The hardest question in k-means clustering is the simplest to ask: how many clusters? You cannot just ask the algorithm. Request K=7 and it will cheerfully carve your data into seven groups, even when there are clearly only three. The elbow method gives you a way to spot the natural number. You plot K against inertia, which is the total of squared distances from every point to its own centroid. As K rises, inertia always falls (more centers means every point sits closer to one). The trick is to find the "elbow", the K where the fall suddenly flattens out. Past that point, adding clusters buys you almost nothing.

Think of packing for a trip. Going from one bag to two makes a huge difference: now your shoes are not crushing your shirts. Two bags to three still helps a bit. But bag eight versus bag nine? You are just splitting hairs. The elbow is the bag count where extra bags stop earning their keep.

📄 elbow_method.py: finding the natural K

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
import numpy as np

# Generate data with 4 natural clusters
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)

# Try K from 1 to 10
inertias = []
for k in range(1, 11):
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    km.fit(X)
    inertias.append(km.inertia_)

print("K  | Inertia    | Drop from previous")
print("---|------------|-------------------")
for i, inertia in enumerate(inertias):
    k = i + 1
    drop = f"{inertias[i-1] - inertia:.1f}" if i > 0 else "---"
    marker = " <-- elbow" if k == 4 else ""
    print(f"{k:<2} | {inertia:>10.1f} | {drop:>10}{marker}")

▶ Output

K  | Inertia    | Drop from previous
---|------------|-------------------
1  |    19780.3 |        ---
2  |     9211.2 |    10569.0
3  |     1919.4 |     7291.8
4  |      362.5 |     1556.9 <-- elbow
5  |      329.3 |       33.2
6  |      294.6 |       34.7
7  |      261.6 |       33.0
8  |      232.0 |       29.6
9  |      209.1 |       22.9
10 |      188.7 |       20.4

What happened here: Look at the "Drop from previous" column, because that is where the elbow lives. Each step from K=1 to K=4 chops the inertia down hard: 10569, then 7292, then a final big cut of 1556.9. Then the cliff ends. From K=5 onward every step removes only about 20 to 35, a flat trickle. That sudden flattening at K=4 is the elbow, and it matches the four groups we built into the data. Real datasets rarely give you a bend this clean, which is exactly why the silhouette score in the next section is a useful second opinion.

Silhouette Analysis for Cluster Quality

Picture yourself at a wedding reception. You feel great when you are seated deep among your own close friends and the next table of strangers is far away. You feel awkward when your chair is jammed right on the boundary between two tables and you could belong to either. The silhouette score measures exactly that comfort level for every data point. It asks a sharper question than inertia: for each point, is it snug inside its own cluster, or is it sitting on the fence next to a neighboring cluster? It compares how close a point is to its own group versus the nearest other group.

Scores run from -1 (this point is in the wrong cluster) through 0 (right on the border) to +1 (deep inside a tight, well-separated cluster). Anything above 0.5 is generally a healthy sign. The handy part: you get one number per K, so you can line them up and compare directly instead of squinting at a bend in a curve.

📄 silhouette_analysis.py: scoring each K, then picking the best

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs

X, _ = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)

# First pass: compute a silhouette score for every candidate K
scores = {}
for k in range(2, 9):
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = km.fit_predict(X)
    scores[k] = silhouette_score(X, labels)

best_k = max(scores, key=scores.get)

print("K  | Silhouette Score")
print("---|------------------")
for k, score in scores.items():
    marker = " <-- best" if k == best_k else ""
    print(f"{k}  | {score:.4f}{marker}")

print(f"\nOptimal K = {best_k} (silhouette = {scores[best_k]:.4f})")

▶ Output

K  | Silhouette Score
---|------------------
2  | 0.6030
3  | 0.7783
4  | 0.8335 <-- best
5  | 0.6976
6  | 0.5882
7  | 0.4487
8  | 0.3322

Optimal K = 4 (silhouette = 0.8335)

What happened here: K=4 wins with a silhouette of 0.83, which confirms the elbow result. Two methods, same answer, and now you can trust it. Watch what happens at K=5: the score drops from 0.83 to 0.70. Forcing a four-group dataset into five groups means k-means has to split one real cluster in half, so the points near that artificial split are no longer confident about which side they belong to, and the average silhouette sags. The score keeps falling as K climbs, because you keep slicing real groups into smaller, blurrier pieces.

Real Example: Customer Segmentation

Here is where k-means clustering earns its salary. Every online store wants to know who its customers really are, but nobody fills in a "VIP" or "about to leave" tag by hand. The standard trick is RFM: describe each customer by three numbers, Recency (how many days since their last order), Frequency (how many orders they have placed), and Monetary (how much they have spent). Feed those three numbers per customer into k-means and let it find the segments. Notice we scale the features first, because k-means measures distance and spend runs into the thousands while frequency is single digits. Without scaling, money would drown out everything else.

📄 customer_segmentation.py: RFM clustering with scikit-learn

import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# Simulated RFM (Recency, Frequency, Monetary) customer data
rng = np.random.default_rng(42)
n = 200

data = {
    "customer_id": [f"C{i:04d}" for i in range(n)],
    "recency_days": np.concatenate([
        rng.integers(1, 30, 60),    # recent buyers
        rng.integers(30, 90, 80),   # moderate
        rng.integers(90, 365, 60),  # dormant
    ]),
    "frequency": np.concatenate([
        rng.integers(10, 50, 60),   # high frequency
        rng.integers(3, 15, 80),    # moderate
        rng.integers(1, 4, 60),     # low frequency
    ]),
    "monetary": np.concatenate([
        rng.integers(500, 5000, 60),
        rng.integers(100, 800, 80),
        rng.integers(10, 150, 60),
    ]),
}
df = pd.DataFrame(data)

# Scale features. K-Means is distance-based, so scaling matters.
scaler = StandardScaler()
X_scaled = scaler.fit_transform(df[["recency_days", "frequency", "monetary"]])

# Cluster into 3 segments
km = KMeans(n_clusters=3, random_state=42, n_init=10)
df["segment"] = km.fit_predict(X_scaled)

# Name segments by their characteristics
segment_names = {}
summary = df.groupby("segment")[["recency_days", "frequency", "monetary"]].mean()
for seg in summary.index:
    row = summary.loc[seg]
    if row["monetary"] > 1000 and row["frequency"] > 15:
        segment_names[seg] = "VIP"
    elif row["recency_days"] > 150:
        segment_names[seg] = "Dormant"
    else:
        segment_names[seg] = "Regular"

df["segment_name"] = df["segment"].map(segment_names)

print("Customer Segments:")
for name in ["VIP", "Regular", "Dormant"]:
    subset = df[df["segment_name"] == name]
    print(f"  {name}: {len(subset)} customers, "
          f"avg recency={subset['recency_days'].mean():.0f}d, "
          f"avg frequency={subset['frequency'].mean():.1f}, "
          f"avg spend=${subset['monetary'].mean():.0f}")

▶ Output

Customer Segments:
  VIP: 54 customers, avg recency=16d, avg frequency=29.7, avg spend=$2909
  Regular: 101 customers, avg recency=67d, avg frequency=7.9, avg spend=$439
  Dormant: 45 customers, avg recency=257d, avg frequency=1.9, avg spend=$79

What happened here: K-means split 200 customers into three segments that actually mean something, using nothing but buying behavior. The VIPs ordered recently (16 days ago on average), order often (almost 30 times), and spend big (around $2,900). The Dormant group has not bought in roughly eight months and barely spends. The Regular group sits in the middle. We never told the algorithm what a VIP is; it found the groups, and we read the averages afterward to put names on them.

This is exactly how a marketing team decides who gets a loyalty perk and who gets a "we miss you" email. One honest caveat: because k-means draws hard boundaries, the counts (54, 101, 45) will not perfectly match the 60, 80, 60 we used to generate the data. Customers near the edges get pulled into whichever cluster center is closest, and that is the right behavior, not a bug.

Common Mistakes

Mistake 1: Forgetting to scale features

This is the number one k-means bug, and it is silent. K-means adds up squared differences across every feature to measure distance. If one feature is salary (tens of thousands) and another is age (a number under 35), the squared salary differences are millions of times larger, so age contributes essentially nothing. The algorithm ends up clustering on salary alone while you think it used both. Always run StandardScaler first.

❌ Wrong: raw features, salary silently dominates

from sklearn.cluster import KMeans

# salary is ~50000, age is ~28. Distance is basically just salary.
X_raw = [[50000, 28], [52000, 31], [48000, 26], [95000, 41]]
km = KMeans(n_clusters=2, random_state=42, n_init=10).fit(X_raw)
# Clusters split on salary only. Age never had a say.

✅ Correct: scale first, every feature counts

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X_raw = [[50000, 28], [52000, 31], [48000, 26], [95000, 41]]
X_scaled = StandardScaler().fit_transform(X_raw)  # zero mean, unit variance
km = KMeans(n_clusters=2, random_state=42, n_init=10).fit(X_scaled)
# Now salary and age are on the same footing.

Why: StandardScaler rescales each feature to zero mean and unit variance, so a one-step move in salary counts the same as a one-step move in age. MinMaxScaler works too, but StandardScaler copes better with outliers, which is why it is the usual first choice before k-means.

Mistake 2: Picking K with no evidence

Hard-coding n_clusters=5 because five feels right is guessing, not analysis. K-means will obey and hand you five groups whether or not five exist. Always back your K with the elbow method, the silhouette score, or (best) both agreeing, the way they both pointed to K=4 above. If the two methods disagree, that is a signal your clusters are not cleanly separated, and it is worth knowing before you build a business decision on top of them.

Practice Exercises

1. Exercise 1: Take the customer segmentation script and rerun the elbow method on the scaled RFM features for K from 1 to 8. Does the elbow agree that 3 segments is the right call, or does it suggest a different number?
2. Exercise 2: Modify the from-scratch kmeans function so it also returns the inertia (the sum of squared distances from each point to its centroid). Confirm your number is close to scikit-learn's km.inertia_ on the same data.
3. Exercise 3: Generate two clusters shaped like long thin stripes (use make_blobs then stretch one axis), run k-means with K=2, and plot the result. K-means assumes round clusters, so watch how it fails on stretched shapes. This is the exact problem the next post solves with DBSCAN.

Conclusion

You started with a pile of unlabeled points and ended with meaningful groups, no answer key required. Along the way you saw the four-step loop that powers k-means clustering, worked one assign-and-update pass by hand on six numbers, used the elbow method and silhouette score together to pick the right K, and turned raw RFM data into named customer segments a marketing team could act on tomorrow. The two habits worth carrying forward: always scale your features first, and never trust a K you cannot back with evidence.

Next up is Hierarchical Clustering and DBSCAN, which handle the stretched, oddly shaped, and unknown-count clusters where k-means quietly struggles. To see how this fits the bigger picture, from Python basics through the full machine learning track, head back to the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on K-Means Clustering

Try each one aloud first. The phrasing you produce under mild pressure is what interviews measure.

Q: Why must you scale features before running k-means, and what happens if you forget?

K-means measures similarity with Euclidean distance, which sums squared differences across every feature. If one feature has a large range (say monetary spend in the thousands) and another is small (frequency in single digits), the large feature dominates the distance and effectively decides the clusters alone. Running StandardScaler first puts every feature on zero mean and unit variance so each one gets an equal vote. Forgetting to scale is the single most common silent bug in clustering pipelines.

Q: What is inertia, and why can it not be used by itself to choose K?

Inertia is the sum of squared distances from every point to its assigned centroid, so it measures how tight the clusters are. The problem is that inertia always decreases as K grows, reaching zero when every point is its own cluster. Because lower is always "better," you cannot just minimize it. The elbow method inspects the rate of decrease and looks for the K where the drop suddenly flattens, and the silhouette score gives an independent second opinion.

Q: What does the silhouette score range mean, and what counts as a good value?

The silhouette score runs from -1 to +1. A value near +1 means a point sits deep inside a tight, well-separated cluster; near 0 means it is on the border between two clusters; and negative means it was likely assigned to the wrong cluster. As a rule of thumb, an average score above 0.5 signals healthy, well-separated groups. You compute it per candidate K and pick the K with the highest average score.

Q: Your elbow plot and silhouette score point to different values of K. How do you decide?

Disagreement is itself information: it usually means your clusters are not cleanly separated, so no single K is obviously right. First confirm you scaled the features and used a fixed random_state with enough n_init restarts. Then let the business context break the tie, since a segmentation with three actionable groups can be more useful than a mathematically tighter five. If neither number is convincing, that is a hint the data may not be spherical, and a density-based method like DBSCAN may fit better.

Q: You run k-means on a dataset with several million rows and memory spikes hard. What do you check and change first?

The standard KMeans computes distances from all points to all centroids each iteration, and with a large n_init it repeats the whole fit many times, so memory and time climb quickly. Switch to MiniBatchKMeans, which updates centroids from small random batches and uses a fraction of the memory for nearly the same result. Also confirm you are not accidentally materializing a giant pairwise distance matrix yourself, and reduce n_init once k-means++ initialization is giving stable centers.

Q: A colleague clusters customers once and reuses the same centroids for months. Why is that risky?

Customer behavior drifts: new buyers arrive, VIPs go dormant, and spend patterns shift with seasons, so centroids fitted on old data slowly stop matching reality. A point flagged "Regular" in January may truly be "Dormant" by June, yet static centroids will keep mislabeling it. The fix is to refit on a schedule (or monitor cluster drift) and to keep the scaler fitted on the same window as the model, otherwise the features feeding prediction no longer match the ones the centroids were learned on.

Series: Python + AI/ML Cookbook. Part 5: Machine Learning

Go deeper: when you outgrow this post, scikit-learn documentation is the next stop.

Related Posts

Previous: ML: Naive Bayes, Probabilistic Classification

Next: ML: Hierarchical Clustering & DBSCAN

Series Home: Python + AI/ML Tutorial Series