Python Naive Bayes is one of the fastest ways to build a working classifier, and this guide explains it from the ground up: Bayes’ theorem in plain English, the “naive” independence assumption, the Gaussian, Multinomial, and Bernoulli variants, and why such a simple algorithm still wins at text classification and spam filtering.

“Naive Bayes should not work as well as it does. The independence assumption is almost always wrong, yet the classifier is almost always competitive.”

Pedro Domingos, The Master Algorithm

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0 | Difficulty: Intermediate | Reading Time: 14 minutes

Here is the everyday version first. A doctor sees a patient who is coughing. Before the cough, the doctor already has a rough sense of how common the flu is this week. That starting hunch is the “prior”. The cough is new evidence, so the doctor updates the hunch: flu now looks more likely than it did a second ago. That updated hunch is the “posterior”. Bayes’ theorem is just the math for doing that update cleanly, and Naive Bayes is what you get when you run the same update for every class (spam or not spam, flu or no flu) and pick whichever class comes out on top.

So where does the word “naive” come from? It is the one shortcut the algorithm takes. Naive Bayes assumes every feature is independent of the others, given the class. In spam detection that means it pretends the word “free” tells you nothing extra once you already know “money” showed up. That is plainly false (spammy words travel in packs), yet the classifier still works well. The exact probability numbers it prints are often too confident, but the ranking, that is, which class wins, is usually right. And the ranking is all you need to make a prediction.

Why bother with such an old, simple algorithm? Because it trains fast (a fraction of a second even on thousands of word features), it copes with sparse high-dimensional text data that makes heavier models crawl, and it learns from very little data. Gmail’s early spam filter leaned on Naive Bayes. To this day it is the first baseline most people reach for on any text classification task.

Prerequisites

Bayes’ Theorem With Real Numbers

Read the diagram top to bottom. A new email arrives with the words “free money”. Naive Bayes combines the prior (how common spam is overall) with the likelihood (how often “free” and “money” show up in spam) to compute a posterior: the probability the email is spam given those words. The naive assumption, that the words are independent given the class, is the trick that keeps the math simple. Instead of modelling how words interact, it just multiplies each word’s probability together. That shortcut is rarely true in real text, yet it works well enough that Naive Bayes remains a top pick for spam detection and text classification.

📄 bayes_theorem.py: spam detection by hand

# Bayes' theorem: P(spam|words) = P(words|spam) * P(spam) / P(words)

# Our email dataset statistics:
# P(spam) = 0.30 (30% of emails are spam, the prior)
# P(not spam) = 0.70

# Word probabilities:
# P("free"|spam) = 0.60      P("free"|not spam) = 0.05
# P("money"|spam) = 0.50     P("money"|not spam) = 0.02

# New email contains: "free money"
# The naive assumption: P("free","money"|spam) = P("free"|spam) * P("money"|spam)

p_spam = 0.30
p_not_spam = 0.70

# Likelihood of "free money" given each class
p_words_given_spam = 0.60 * 0.50  # = 0.30
p_words_given_not_spam = 0.05 * 0.02  # = 0.001

# Evidence (total probability of seeing "free money")
p_words = p_words_given_spam * p_spam + p_words_given_not_spam * p_not_spam

# Posterior probability
p_spam_given_words = (p_words_given_spam * p_spam) / p_words
p_not_spam_given_words = (p_words_given_not_spam * p_not_spam) / p_words

print("Bayes' Theorem for 'free money':")
print(f"  P(words|spam) = {p_words_given_spam:.4f}")
print(f"  P(words|not spam) = {p_words_given_not_spam:.4f}")
print(f"  P(evidence) = {p_words:.6f}")
print(f"")
print(f"  P(spam|'free money') = {p_spam_given_words:.4f} ({p_spam_given_words:.1%})")
print(f"  P(not spam|'free money') = {p_not_spam_given_words:.4f} ({p_not_spam_given_words:.1%})")
print(f"")
print(f"  Prediction: {'SPAM' if p_spam_given_words > 0.5 else 'NOT SPAM'}")
print(f"  The prior was 30% spam. After seeing 'free money', it is {p_spam_given_words:.1%}.")

▶ Output

Bayes' Theorem for 'free money':
  P(words|spam) = 0.3000
  P(words|not spam) = 0.0010
  P(evidence) = 0.090700

  P(spam|'free money') = 0.9923 (99.2%)
  P(not spam|'free money') = 0.0077 (0.8%)

  Prediction: SPAM
  The prior was 30% spam. After seeing 'free money', it is 99.2%.

What happened here: Walk the numbers yourself. Spam makes “free money” likely (0.60 times 0.50 = 0.30), while normal email almost never does (0.05 times 0.02 = 0.001). Bayes’ theorem weighs each side by how common it is and divides by the total, which gives 0.9923. So before reading the email, there was a 30% chance it was spam. After seeing just two words, that jumped to 99.2%. That huge swing is the whole point of Bayes’ theorem: evidence that clearly separates the classes moves the probability a long way in one step.

Three Variants: Gaussian, Multinomial, Bernoulli

The three Python Naive Bayes variants are the same idea wearing different clothes. Each one assumes a different shape for your data, so you pick the one that matches what you have. Gaussian expects continuous numbers that bunch up around an average, like age or salary. Multinomial expects counts, such as how many times each word appears in an email. Bernoulli expects plain yes/no flags, like “did this word show up at all”. Think of it as choosing the right adapter plug: same appliance, different socket. Match the variant to your data and you are done.

📄 nb_variants.py: when to use each variant

import numpy as np
from sklearn.naive_bayes import GaussianNB, MultinomialNB, BernoulliNB
from sklearn.datasets import make_classification, fetch_20newsgroups
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline

# 1. GaussianNB: for continuous features
X_cont, y_cont = make_classification(n_samples=500, n_features=10, random_state=42)
gnb_scores = cross_val_score(GaussianNB(), X_cont, y_cont, cv=5)
print(f"GaussianNB (continuous features): {gnb_scores.mean():.3f}")

# 2. MultinomialNB: for text / count data
texts = [
    "free money click now", "win prize congratulations", "meeting tomorrow 3pm",
    "project deadline friday", "cheap offers discount sale", "code review pull request",
    "winner selected lottery", "quarterly report attached", "free trial limited time",
    "standup notes sprint planning"
]
labels = [1, 1, 0, 0, 1, 0, 1, 0, 1, 0]  # 1=spam, 0=not spam

pipe = Pipeline([
    ("vectorizer", CountVectorizer()),
    ("classifier", MultinomialNB())
])
pipe.fit(texts, labels)

test_emails = ["free discount offer", "project meeting agenda", "win money now"]
for email in test_emails:
    pred = pipe.predict([email])[0]
    proba = pipe.predict_proba([email])[0]
    print(f"  '{email}' → {'Spam' if pred else 'Not spam'} (P={proba[pred]:.2f})")

# 3. BernoulliNB: for binary features (word present/absent)
pipe_bn = Pipeline([
    ("vectorizer", CountVectorizer(binary=True)),  # binary=True!
    ("classifier", BernoulliNB())
])
pipe_bn.fit(texts, labels)
pred = pipe_bn.predict(["free money click"])
print(f"\nBernoulliNB (binary): 'free money click' → {'Spam' if pred[0] else 'Not spam'}")

▶ Output

GaussianNB (continuous features): 0.890
  'free discount offer' → Spam (P=0.85)
  'project meeting agenda' → Not spam (P=0.81)
  'win money now' → Spam (P=0.88)

BernoulliNB (binary): 'free money click' → Spam

What happened here: All three variants did their job on the data shape they expect. GaussianNB scored 0.890 on the continuous features. MultinomialNB read the word counts and sorted the three test emails into spam and not-spam, with probabilities around 0.81 to 0.88 (confident, but not the wild 0.99 you sometimes see on bigger vocabularies). BernoulliNB looked only at whether each word was present and still flagged “free money click” as spam. The takeaway: for text, reach for MultinomialNB first. For plain tabular data with continuous columns, GaussianNB is a quick, no-fuss baseline.

Real Text Classification Example

Picture a librarian who sorts returned books onto the right shelves by glancing at a few telltale words on each cover. She does not read every page, she just reacts to words that lean one way (“orbit” says space, “compiler” says computing) and drops the book on the likeliest shelf. That is exactly what Python Naive Bayes does below: it scans newsgroup posts for words, weighs which topic those words favour, and files each post under the winning category.

📄 text_classification.py: classifying newsgroup posts

from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score, classification_report
import time

# Load 4 categories of newsgroup posts
categories = ["sci.space", "comp.graphics", "rec.sport.baseball", "talk.politics.guns"]

train = fetch_20newsgroups(subset="train", categories=categories, random_state=42)
test = fetch_20newsgroups(subset="test", categories=categories, random_state=42)

# Pipeline: TF-IDF vectorization + Naive Bayes
pipe = Pipeline([
    ("tfidf", TfidfVectorizer(max_features=10000, stop_words="english")),
    ("nb", MultinomialNB(alpha=0.1))
])

start = time.time()
pipe.fit(train.data, train.target)
train_time = time.time() - start

start = time.time()
predictions = pipe.predict(test.data)
pred_time = time.time() - start

accuracy = accuracy_score(test.target, predictions)
print(f"Accuracy: {accuracy:.1%}")
print(f"Training time: {train_time:.3f}s")
print(f"Prediction time: {pred_time:.3f}s")
print(f"Samples: {len(train.data)} train, {len(test.data)} test")
print(f"Features: {pipe.named_steps['tfidf'].get_feature_names_out().shape[0]} words")
# sklearn sorts the categories alphabetically, so use train.target_names, not your own list
print(f"\n{classification_report(test.target, predictions, target_names=train.target_names)}")

▶ Output

Accuracy: 96.2%
Training time: 0.783s
Prediction time: 0.404s
Samples: 2320 train, 1544 test
Features: 10000 words

                    precision    recall  f1-score   support

     comp.graphics       0.94      0.95      0.94       389
rec.sport.baseball       0.98      0.98      0.98       397
         sci.space       0.96      0.94      0.95       394
talk.politics.guns       0.97      0.98      0.98       364

          accuracy                           0.96      1544
         macro avg       0.96      0.96      0.96      1544
      weighted avg       0.96      0.96      0.96      1544

What happened here: 96.2% accuracy sorting newsgroup posts into 4 topics, with training and prediction together finishing in well under two seconds on 10,000 text features. (The exact times shift run to run, and most of that second goes into building the TF-IDF (Term Frequency-Inverse Document Frequency) matrix, not fitting the model. The fit itself is the cheap part.) That speed is the whole appeal of Naive Bayes: cost grows roughly in line with the number of features and samples, nothing worse.

One note on reading the report: scikit-learn sorts the requested categories alphabetically, no matter what order you passed them in, so always hand classification_report the dataset’s own train.target_names rather than your original list, or the rows get attached to the wrong labels. For text with thousands of word features, Naive Bayes is genuinely hard to beat on the speed-versus-accuracy trade.

When to Choose Python Naive Bayes

📄 nb_vs_others.py: Naive Bayes vs other classifiers

from sklearn.naive_bayes import GaussianNB
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score
from sklearn.datasets import make_classification
import time
import numpy as np

X, y = make_classification(n_samples=2000, n_features=20, random_state=42)

models = {
    "Naive Bayes": GaussianNB(),
    "Logistic Regression": LogisticRegression(max_iter=200),
    "Random Forest": RandomForestClassifier(n_estimators=100, random_state=42),
    "SVM (RBF)": SVC(),
}

print(f"{'Model':<22} | {'CV Score':>10} | {'Time (s)':>10}")
print("-" * 47)
for name, model in models.items():
    start = time.time()
    scores = cross_val_score(model, X, y, cv=5)
    elapsed = time.time() - start
    print(f"{name:<22} | {scores.mean():>9.3f} | {elapsed:>9.3f}")

print(f"\nNaive Bayes: fastest, good enough for many tasks.")
print(f"Use it when speed matters, data is sparse, or as a baseline.")

▶ Output

Model                  |   CV Score |   Time (s)
-----------------------------------------------
Naive Bayes            |     0.888 |     0.048
Logistic Regression    |     0.879 |     0.073
Random Forest          |     0.902 |     4.871
SVM (RBF)              |     0.894 |     0.454

Naive Bayes: fastest, good enough for many tasks.
Use it when speed matters, data is sparse, or as a baseline.

What happened here: This is the Domingos quote at the top of the page, in numbers. Random Forest came out slightly ahead on accuracy (0.902), but Naive Bayes (0.888) actually edged out Logistic Regression and finished roughly a hundred times faster than the forest. The cross-validation (CV) scores are exact and repeatable because everything is seeded; the times are not, so treat them as ballpark (Random Forest in particular swings by several seconds run to run). The pattern holds anyway: Naive Bayes rarely wins the accuracy contest outright, yet it lands close to the pack for a sliver of the cost. That is exactly why it makes such a good first baseline.

Common Mistakes

❌ Mistake: Trusting Naive Bayes probability values

# Naive Bayes outputs "probabilities" that are often extreme (0.99 or 0.01)
# because the independence assumption inflates/deflates them.
# The RANKING is reliable (higher probability = more likely correct class)
# but the actual values are poorly calibrated.

# If you need calibrated probabilities, use CalibratedClassifierCV:
from sklearn.calibration import CalibratedClassifierCV
from sklearn.naive_bayes import GaussianNB

# calibrated_nb = CalibratedClassifierCV(GaussianNB(), cv=5, method="isotonic")
# This wraps NB and calibrates its probability outputs.
print("Naive Bayes predictions (which class wins) are reliable.")
print("Naive Bayes probabilities (exact numbers) are NOT well-calibrated.")
print("Use CalibratedClassifierCV if you need accurate probabilities.")

Practice Exercises

1. Exercise 1: Build a spam filter. Collect a handful of spam and normal messages, run them through CountVectorizer plus MultinomialNB, and test it on three new messages you write yourself.
2. Exercise 2: Run the same dataset through all three variants (Gaussian, Multinomial, Bernoulli) and compare cross-validation scores. Notice which variant fits text best and why.
3. Exercise 3: Code Multinomial Naive Bayes from scratch with NumPy, including Laplace smoothing (the alpha count), then check your predictions match scikit-learn’s MultinomialNB on the same email data.

Conclusion

You now have the full picture of Python Naive Bayes: Bayes’ theorem as a hunch that gets updated by evidence, the “naive” independence shortcut that keeps the math cheap, and the three variants (Gaussian for continuous numbers, Multinomial for word counts, Bernoulli for present/absent flags). You saw it hit 96% accuracy on newsgroup topics in under two seconds, and you saw why its exact probabilities are best trusted for their ranking, not their raw values. When you need a fast, honest baseline for text or sparse data, this is the first tool to reach for.

Next up is K-Means Clustering, where we leave labelled data behind and let the algorithm discover groups on its own. For the full path from Python basics through machine learning, head to the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on Naive Bayes

Interviewers rarely ask for definitions. They ask what happens in situations like these.

Q: In one sentence, what does the “naive” in Naive Bayes actually assume, and why is it wrong?

It assumes every feature is conditionally independent of the others given the class, so the joint likelihood is just the product of each feature’s likelihood. In real text that is false because words travel together (the word “money” makes “free” more likely, not independent). The assumption is wrong, but because the class ranking still usually comes out correct, the classifier stays accurate.

Q: Which Naive Bayes variant would you pick for TF-IDF text features, raw word counts, and continuous sensor readings?

Use MultinomialNB for raw word counts and it also handles TF-IDF weights fine in practice. For continuous readings like temperature or salary, use GaussianNB, which models each feature as a bell curve per class. BernoulliNB is the odd one out: reach for it when features are binary present/absent flags rather than counts.

Q: What is Laplace (alpha) smoothing and what breaks without it?

Smoothing adds a small count (alpha, default 1.0) to every feature count so that no probability is ever exactly zero. Without it, a single word that never appeared in a class during training gives that class a likelihood of zero, and since the likelihoods are multiplied, one zero wipes out the whole posterior. Lower alpha means less smoothing (sharper, riskier estimates); higher alpha means more smoothing (safer, blunter estimates).

Q: Why is Naive Bayes so fast to train compared to Random Forest or Support Vector Machine (SVM)?

Training is essentially one pass to count frequencies (or compute means and variances for GaussianNB), so cost grows linearly with the number of samples times features and there is no iterative optimisation. There are no trees to grow and no support vectors to solve for. That is why in the comparison above it finished roughly a hundred times faster than Random Forest while landing close on accuracy.

Q: Scenario: your spam model reports 99.8% confidence on almost every email, including ones it gets wrong. What is going on and what do you do?

This is the classic Naive Bayes calibration problem: the independence assumption multiplies many correlated word probabilities together, so the posterior gets pushed toward 0 or 1 and the raw number is overconfident. The class it picks is usually still right, so trust the ranking, not the value. If a downstream system needs a trustworthy probability (for example a threshold or a cost decision), wrap the model in CalibratedClassifierCV with isotonic or sigmoid calibration.

Q: Scenario: you deploy a text classifier and it crashes at prediction time on a message full of words it never saw in training. Where do you look first?

First check that the same fitted vectorizer from training is used at prediction, not a fresh one, because a new CountVectorizer produces a different vocabulary and column layout. Unseen words should simply be ignored by a properly fitted vectorizer, not cause a crash, so a shape mismatch usually points to re-fitting or a broken pipeline. Using a single sklearn Pipeline (vectorizer plus classifier) is the clean fix, since it guarantees the exact same transform at train and predict time.

Q: When would you not use Naive Bayes?

Avoid it when feature interactions carry most of the signal, since the independence assumption throws that information away and a tree ensemble or gradient boosting model will do much better. It is also a poor fit when you genuinely need well-calibrated probabilities out of the box. For dense tabular data with strongly correlated features, logistic regression or Random Forest is usually the stronger baseline.

Series: Python + AI/ML Cookbook, Part 5: Machine Learning

Want more? scikit-learn documentation documents everything this post could not fit.

Related Posts

Previous: KNN in Python: K-Nearest Neighbors, Distance-Based Classification

Next: ML: K-Means Clustering, Unsupervised Discovery

Series Home: Python + AI/ML Tutorial Series