A Python recommender system learns your taste from patterns in the numbers, and this walkthrough builds one from scratch: collaborative filtering (people who liked what you liked also liked X), content-based filtering (here are items similar to ones you already enjoyed), SVD (Singular Value Decomposition) matrix factorization, and the cold start problem that trips up every brand new system.

“For two decades now, Amazon.com has been building a store for every customer.”

Greg Linden, early Amazon recommendations engineer

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0 | Difficulty: Advanced | Reading Time: 13 minutes

Every time Netflix suggests a show, Spotify builds a playlist, or Amazon says “customers who bought this also bought that,” a recommender system is quietly doing the work. Here is the part that surprises people. The system has never watched a single movie and does not know what “action” or “comedy” even means. It just looks at who rated what, spots patterns in the numbers, and guesses the rest. That is the whole trick, and this post takes it apart piece by piece.

Think about how a good friend recommends a film. They do not read a plot summary database. They just know that you and another friend both loved the same three movies, so when that friend raves about a fourth one, they pass it along to you. That is collaborative filtering in one sentence: similar people tend to like similar things. Content-based filtering works the other way around, like a librarian who says “you liked this thriller, so here are more thrillers with the same tags.” Hybrid systems use both.

The catch underneath all of it is a giant, mostly empty table: millions of users, millions of items, and almost every cell blank because nobody has time to rate everything. The job is to fill in those blanks with smart guesses.

Prerequisites

Python Recommender System Architecture

Imagine picking a restaurant for dinner. You could ask a friend whose taste matches yours where they just ate (that is collaborative filtering), or you could scan a menu and choose a place because it serves the paneer dishes you already love (that is content-based). The diagram splits the two main roads to a recommendation. Collaborative filtering reads the ratings table: it finds users (or items) that behave alike and borrows their opinions.

Content-based filtering reads the item itself: genre, tags, description, and matches that against what you already enjoyed. Hybrid systems take the best of both, leaning on collaborative signals when a user has plenty of history and falling back on content features for brand new items with no ratings yet. This is the first real design decision you make. Collaborative filtering needs interaction data (ratings, clicks, watches), while content-based needs item features (genre, description). The code below builds both halves of a Python recommender system by hand so you can see exactly what each one is doing.

📄 collaborative_filtering.py: user-based and item-based similarity

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity

# User-item rating matrix (0 = not rated)
ratings = np.array([
    #  Movie1 Movie2 Movie3 Movie4 Movie5
    [5,     3,     0,     1,     4],  # Rahul
    [4,     0,     0,     1,     5],  # Aditi
    [1,     1,     0,     5,     0],  # Niranjan
    [0,     0,     5,     4,     0],  # Aviraj
    [0,     1,     4,     5,     0],  # Anvi
])

users = ["Rahul", "Aditi", "Niranjan", "Aviraj", "Anvi"]
movies = ["Movie1", "Movie2", "Movie3", "Movie4", "Movie5"]

# User-user similarity (cosine similarity)
user_sim = cosine_similarity(ratings)
print("User-User Similarity:")
for i, u in enumerate(users):
    similar = np.argsort(user_sim[i])[::-1][1]  # most similar user (not self)
    print(f"  {u} is most similar to {users[similar]} (sim={user_sim[i, similar]:.3f})")

# Recommend for Rahul (user 0): predict his Movie3 rating
# Using weighted average of similar users who rated Movie3
target_user = 0
target_item = 2  # Movie3 (0-indexed)

# Find users who rated Movie3
rated_mask = ratings[:, target_item] > 0
if rated_mask.sum() > 0:
    sim_scores = user_sim[target_user, rated_mask]
    user_ratings = ratings[rated_mask, target_item]
    predicted = np.dot(sim_scores, user_ratings) / (np.abs(sim_scores).sum() + 1e-8)
    print(f"\nPredicted rating for {users[target_user]} on {movies[target_item]}: {predicted:.1f}")

▶ Output

User-User Similarity:
  Rahul is most similar to Aditi (sim=0.886)
  Aditi is most similar to Rahul (sim=0.886)
  Niranjan is most similar to Anvi (sim=0.772)
  Aviraj is most similar to Anvi (sim=0.964)
  Anvi is most similar to Aviraj (sim=0.964)

Predicted rating for Rahul on Movie3: 4.3

What happened here: Our five sample users are Rahul, Aditi, Niranjan, Aviraj, and Anvi, each one a row of movie ratings. Rahul and Aditi come out most similar (0.886 cosine similarity) because their rating rows point the same way: both score Movie1 and Movie5 high and rate Movie4 low. Niranjan, Aviraj, and Anvi lean toward Movie3 and Movie4 instead, so they sit in a different corner of the table. Cosine similarity is just measuring the angle between two rows of numbers, so two people who rate things in the same shape land close together even if one is a touch more generous overall.

The predicted 4.3 for Rahul on Movie3 is a weighted average of the people who actually rated Movie3, each vote counted in proportion to how much they look like Rahul. It lands high because the users who liked Movie3 (Aviraj rated it 5 and Anvi rated it 4) are not wildly far from Rahul in this tiny dataset. With only five users the numbers are noisy, which is exactly why real systems need thousands of ratings before the guesses settle down.

Matrix Factorization with SVD

Comparing whole rows of ratings works, but it gets slow and shaky when the table is mostly empty. Matrix factorization takes a different angle. It assumes a handful of hidden “tastes” drive every rating, things like “how much action do you want” or “do you prefer mainstream hits or niche stuff,” and it tries to score each user and each movie on those hidden dials. Picture a music app. Behind the scenes it might decide every song has a secret “energy” number and a secret “mood” number, and every listener has a preference on those same two dials.

Multiply the listener’s dials against a song’s dials and you get a predicted rating. You never told it those dials existed. SVD is the math that pulls them straight out of the numbers.

One honest catch first. SVD needs a full table with no holes, but our ratings table is full of zeros for “not rated yet.” So before factorizing we fill each blank with that user’s average rating. That is the simplest possible patch, and it works for a demo, but it quietly nudges the results, especially for someone who has only rated one or two movies. Watch for that when you read the output.

📄 svd_recommender.py: latent factor discovery

import numpy as np

# Same rating matrix, but fill missing with row means
ratings = np.array([
    [5, 3, 0, 1, 4],
    [4, 0, 0, 1, 5],
    [1, 1, 0, 5, 0],
    [0, 0, 5, 4, 0],
    [0, 1, 4, 5, 0],
], dtype=float)

# Fill zeros with row mean (simple imputation)
for i in range(ratings.shape[0]):
    mask = ratings[i] > 0
    if mask.sum() > 0:
        ratings[i][~mask] = ratings[i][mask].mean()

# SVD decomposition
U, sigma, Vt = np.linalg.svd(ratings, full_matrices=False)

# Keep top 2 latent factors
k = 2
U_k = U[:, :k]
S_k = np.diag(sigma[:k])
Vt_k = Vt[:k, :]

# Reconstruct approximate rating matrix
ratings_approx = U_k @ S_k @ Vt_k

users = ["Rahul", "Aditi", "Niranjan", "Aviraj", "Anvi"]
movies = ["Movie1", "Movie2", "Movie3", "Movie4", "Movie5"]

print("Reconstructed ratings (2 latent factors):")
for i, u in enumerate(users):
    row = " ".join(f"{r:>5.1f}" for r in ratings_approx[i])
    print(f"  {u:<12} {row}")

print(f"\nLatent factors explain how users and items relate.")
print(f"Factor 1 might capture action vs drama preference.")
print(f"Factor 2 might capture mainstream vs niche taste.")

▶ Output

Reconstructed ratings (2 latent factors):
  Rahul          4.4   3.4   3.4   0.9   4.2
  Aditi          4.5   3.5   3.5   1.1   4.3
  Niranjan       1.3   0.7   2.7   4.9   2.0
  Aviraj         4.9   3.6   4.8   3.9   5.1
  Anvi           2.8   1.9   3.8   5.1   3.4

Latent factors explain how users and items relate.
Factor 1 might capture action vs drama preference.
Factor 2 might capture mainstream vs niche taste.

What happened here: SVD squeezed the whole table down to two hidden factors and then rebuilt it. Look at Rahul and Aditi: their rebuilt rows are almost identical (high on Movie1 and Movie5, low on Movie4), which is the model saying "these two have the same taste." Niranjan and Anvi both come out high on Movie4, so they share a different taste. Nobody labeled those factors, the model just found them in the numbers.

Now look at Aviraj, whose row reads high on nearly everything. That is the row-mean patch leaking through: in the original table Aviraj had rated only two movies, so most of his row was filled with his own average, and SVD had very little real signal to work with. This is the honest limit of the demo. The shape is right for users with enough ratings, and noisy for users with almost none.

That gap is exactly why production systems use smarter factorization (it learns only from the cells that were actually filled in) instead of patching the holes first.

Common Mistakes

The single biggest trap in any Python recommender system is the cold start problem. Collaborative filtering only works when it has ratings to chew on. A brand new user has rated nothing, and a brand new movie has been rated by nobody, so the similarity math has nothing to compare. It is like asking a new coworker who they would recommend for a project on their first morning, before they have met anyone. The fix is to lean on something other than ratings until the ratings show up.

❌ Mistake: ignoring the cold start problem

# New users have no ratings -> collaborative filtering cannot work.
# New items have no ratings -> cannot compute item similarity.
# Solutions:
# 1. Ask new users for a few ratings (onboarding quiz)
# 2. Use content-based filtering for new items (item features)
# 3. Popularity-based fallback (recommend most popular)
print("Cold start solutions:")
print("  New users: onboarding quiz or popularity-based recs")
print("  New items: content-based filtering on item metadata")
print("  Hybrid: combine collaborative + content-based")

▶ Output

Cold start solutions:
  New users: onboarding quiz or popularity-based recs
  New items: content-based filtering on item metadata
  Hybrid: combine collaborative + content-based

What happened here: There is no clever algorithm hiding in this snippet, and that is the point. The cold start fix is a product decision, not a math trick. When someone signs up, ask them to tap a few movies they like (that is the onboarding quiz Netflix shows you on day one). For a movie nobody has rated yet, recommend it using its own features (genre, cast, description) until real ratings trickle in. And when you have nothing at all to go on, fall back to plain popularity, because "the thing most people watch" is a safe first guess. A good hybrid system switches between these gracefully instead of showing a new user an empty screen.

Practice Exercises

1. Exercise 1: Switch the collaborative filtering script from user-user to item-item. Run cosine_similarity on ratings.T (the transposed matrix) so you compare movie columns instead of user rows, then print which movie is most similar to each one. Note how the answers differ from the user-based version.
2. Exercise 2: Build a tiny content-based recommender. Give each of the five movies a couple of tag features (for example genre as a one-hot vector), then for a user recommend the unseen movie whose tags are closest to the movies they already rated highly. This is the fallback that survives the cold start.
3. Exercise 3: Combine both. Write a function that returns content-based recommendations when a user has fewer than three ratings and switches to collaborative filtering once they have more. Test it with a brand new user (no ratings) and confirm it still returns something sensible instead of crashing.

Conclusion

You built a Python recommender system from the ground up and saw both roads it can take. Collaborative filtering reads the ratings table and borrows opinions from users who behave like you. Content-based filtering reads the item itself and matches features to your taste. SVD matrix factorization squeezes the whole table down to a handful of hidden "taste" dials and rebuilds it, and you saw exactly where the simple row-mean patch leaks for users with almost no history. Finally, you met the cold start problem and the product-level fixes (onboarding quiz, content fallback, popularity default) that keep a brand new user from staring at an empty screen.

Next up in the series is time series analysis with ARIMA and decomposition, where you predict values that move over time instead of filling in a static ratings table. For the full path from Python basics to advanced machine learning, head to the Python + AI/ML tutorial series home and follow it in order.

Frequently Asked Questions

Interview Questions on Recommender Systems

Scenario questions, not trivia: this is the form this topic takes in a real interview.

Q: Why do recommender systems use cosine similarity instead of plain Euclidean distance on the rating rows?

Cosine similarity measures the angle between two rating vectors, not their raw distance, so it captures whether two people rate things in the same shape even if one is consistently more generous. A harsh rater who gives 2s and 4s and a generous rater who gives 4s and 5s can still be judged similar if their highs and lows line up. Euclidean distance would push them apart just because the absolute numbers differ. That said, mean-centering the ratings first (Pearson correlation) handles the generosity gap even better.

Q: What is the difference between explicit and implicit feedback, and why does it matter?

Explicit feedback is a user directly stating a preference, like a 1-to-5 star rating. Implicit feedback is behavior you infer preference from, such as clicks, watch time, or purchases, where a missing signal does not clearly mean dislike. Implicit data is far more abundant but noisier, and you cannot treat an unwatched item as a negative the way you would treat a low star rating. Production systems usually lean on implicit feedback because most users never rate anything.

Q: What separates memory-based collaborative filtering from model-based collaborative filtering?

Memory-based methods (the user-user and item-item similarity in this post) compute predictions directly from the raw rating table at query time using similarity scores. Model-based methods (like SVD matrix factorization) first learn a compact model, such as latent factors, and then predict from that model. Model-based approaches handle sparsity and scale better because they compress the table, while memory-based approaches are simpler and easier to explain but get slow as the matrix grows.

Q: Your SVD recommender returns inflated, near-identical predictions for a user who has rated only one movie. What do you check first?

Start with how you filled the missing values before factorizing. In the demo we patch blanks with each user's row mean, so a user with a single rating has almost their whole row set to one constant, and SVD just echoes that constant back. The fix is to stop imputing and use a factorization that trains only on the cells that were actually observed (SGD or ALS on the known ratings), or to fall back to content-based and popularity recommendations until that user has enough real ratings to learn from.

Q: In production your recommender shows almost everyone the same handful of popular items and engagement is flat. What is going wrong and what do you check?

This is popularity bias: the model keeps recommending already-popular items, which get more interactions, which makes them look even more popular, a feedback loop that crowds out personalization. Check whether your training signal is dominated by a few blockbuster items, whether cold or niche items ever get surfaced, and whether you have any diversity or exploration built in. Fixes include down-weighting popularity, adding an exploration slice that occasionally surfaces less-seen items, and measuring coverage and diversity alongside accuracy.

Q: How would you scale a recommender to millions of users and millions of items?

Computing full pairwise similarity is quadratic and does not survive at that size, so you move to model-based factorization trained with SGD or ALS, which reduces each user and item to a short latent vector. To serve recommendations fast you precompute item-item neighbors offline and use approximate nearest neighbor search over the item vectors at request time. Heavy lifting (factorization, neighbor tables) runs as batch jobs, while the live path only does a quick lookup and ranking.

Series: Python + AI/ML Cookbook (Part 5: Machine Learning)

Go deeper: the official Python documentation covers every edge case of this topic.

Related Posts

Previous: ML: Anomaly Detection with Isolation Forest

Next: ML: Time Series Analysis with ARIMA and Decomposition

Series Home: Python + AI/ML Tutorial Series