Python sentiment analysis lets you build a working analyzer that reads a movie review and tells you whether it is positive or negative. You will clean the raw text, turn words into numbers with TF-IDF (term frequency times inverse document frequency), train a classifier on 2,000 real movie reviews, and score it honestly on reviews it has never seen. No Graphics Processing Unit (GPU), no deep learning, runs in a few seconds on a laptop.

“Sentiment analysis is the gateway drug of NLP. Once you can label text positive or negative, you start asking why, and that is when the real natural language work begins.”

Christopher Manning, Stanford NLP

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0, NLTK 3.9.4 | Difficulty: Intermediate | Reading Time: 19 minutes

Here is the problem. Your product has 8,000 reviews and your boss wants to know, by Friday, how many are angry. Reading them one by one is a week of work. Sentiment analysis hands that job to a model: feed it the text, get back “positive” or “negative” plus a confidence number. Support teams use it to push the furious tickets to the top of the queue. Marketing uses it to measure how a campaign landed. Product teams use it to find what people quietly hate.

Think of it like a recipe rating jar at a restaurant. Every guest drops a note, and at the end of the night the owner sorts them into two piles, happy and unhappy, without reading every word. A sentiment classifier is that sorting reflex, automated. In this recipe we build one the classic way: clean the text, score the words with TF-IDF, and train logistic regression. This approach trains in seconds, needs no GPU, and is easy to debug. When you truly need top accuracy, transformer models such as BERT or RoBERTa do better, but they cost orders of magnitude more to run, so the classic pipeline is still where most teams start.

The Quickest Possible Classifier

Before any theory, here is the whole idea in fourteen lines. Six tiny reviews, three positive and three negative, and two new sentences the model has never read. This is Python sentiment analysis stripped to the bone so you can see the shape of it.

📄 quick_start.py: the smallest sentiment classifier that works

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

# A tiny, hand-labelled set. 1 = positive, 0 = negative.
reviews = [
    "absolutely loved this movie, fantastic acting",
    "terrible film, boring and a complete waste of time",
    "a brilliant and moving story, highly recommend",
    "awful script, dull plot, fell asleep halfway through",
    "stunning visuals and a powerful soundtrack",
    "worst movie of the year, painful to sit through",
]
labels = [1, 0, 1, 0, 1, 0]

model = Pipeline([
    ("tfidf", TfidfVectorizer(stop_words="english")),
    ("clf", LogisticRegression(random_state=42)),
])
model.fit(reviews, labels)

for text in ["a wonderful and moving film", "boring and a waste of time"]:
    pred = model.predict([text])[0]
    print(f'{"Positive" if pred == 1 else "Negative"}  <- "{text}"')

▶ Output

Positive  <- "a wonderful and moving film"
Negative  <- "boring and a waste of time"

What happened here: The Pipeline glues two steps together. TfidfVectorizer turns each review into a row of numbers, one number per word. LogisticRegression learns which words lean positive (loved, brilliant, stunning) and which lean negative (terrible, awful, worst). On a new sentence the model adds up those leanings and picks a side. Six examples is far too few for real work, so do not trust the confidence here, but the machinery is exactly the same as the production pipeline we build next. We set random_state=42 so you get the same result every run.

Prerequisites

Python Sentiment Analysis Pipeline

The diagram walks the same path our Python sentiment analysis code takes. A raw movie review goes in at the top. We lowercase it and strip noise like HTML tags and punctuation. TF-IDF then turns the cleaned words into a row of numbers. Logistic regression reads that row and outputs a probability. If the probability of “positive” is above 0.5 the review lands in the positive pile, otherwise the negative pile. The feature step (TF-IDF here) is where the classic approach and modern transformers differ most: TF-IDF counts words, while a transformer reads them in context. We build the classic version because it is fast, cheap, and surprisingly hard to beat as a baseline.

Step 1: Clean the Text and Build TF-IDF Features

A model cannot read English. It reads numbers. So before anything else we tidy the text, then convert it to a vector. Cleaning is the boring-but-vital part: lowercase everything so “Great” and “great” count as one word, throw away HTML tags, and drop digits and punctuation that carry no sentiment.

TF-IDF is the clever part. The name stands for term frequency times inverse document frequency. In plain words: a word matters more if it appears often in this review (term frequency) but is rare across all reviews (inverse document frequency). Think of a noisy classroom where everyone is shouting. The word “the” is every kid yelling at once, so you tune it out. The word “brilliant” is one kid who rarely speaks up, so when they do, you listen. TF-IDF gives that rare, telling word a high weight and the everyday words a low one.

📄 features.py: clean text, then score words with TF-IDF

import re
from sklearn.feature_extraction.text import TfidfVectorizer

def clean(text):
    text = text.lower()                       # "Great" and "great" become one token
    text = re.sub(r"<[^>]+>", " ", text)      # strip HTML tags like <br />
    text = re.sub(r"[^a-z\s]", " ", text)     # drop digits and punctuation
    text = re.sub(r"\s+", " ", text).strip()  # squeeze repeated spaces
    return text

raw = "The acting was GREAT!!! <br /> Best movie of 2026, 10/10."
print("Before:", raw)
print("After: ", clean(raw))

# TF-IDF turns cleaned text into a sparse numeric vector
corpus = [
    "the movie was great and the acting was great",
    "the movie was boring and dull",
    "great acting saved a dull script",
]
vec = TfidfVectorizer(stop_words="english")
matrix = vec.fit_transform(corpus)

print("\nVocabulary:", vec.get_feature_names_out().tolist())
print("Matrix shape (docs x features):", matrix.shape)
print("\nWeights for review 1:")
row = matrix[0].toarray()[0]
for word, weight in sorted(
    zip(vec.get_feature_names_out(), row), key=lambda p: -p[1]
):
    if weight > 0:
        print(f"  {word:<8} {weight:.3f}")

▶ Output

Before: The acting was GREAT!!! <br /> Best movie of 2026, 10/10.
After:  the acting was great best movie of

Vocabulary: ['acting', 'boring', 'dull', 'great', 'movie', 'saved', 'script']
Matrix shape (docs x features): (3, 7)

Weights for review 1:
  great    0.816
  acting   0.408
  movie    0.408

What happened here: Cleaning knocked the raw string down to plain lowercase words. Notice the digits and the <br /> tag vanished. Then TF-IDF built a vocabulary of seven words across the three reviews. The common words “the”, “was” and “and” never made it in because stop_words="english" filters out filler. In review 1 the word “great” scored 0.816, the highest, because it appears twice in that short review and is fairly distinctive. The matrix shape (3, 7) means three reviews, each now a row of seven numbers. That numeric grid is exactly what the classifier trains on.

Step 2: Train on 2,000 Real Reviews

Six toy reviews prove the wiring works, but they prove nothing about accuracy. For a real number we need real data. NLTK ships the classic movie reviews corpus: 2,000 full reviews, 1,000 positive and 1,000 negative, hand-labelled by researchers. The first time you use it, download it once with nltk.download("movie_reviews").

The golden rule of honest evaluation: never score a model on data it trained on. That is like grading students with the exact questions you handed out as practice. We split the 2,000 reviews into 1,500 for training and 500 held back for testing. The model never sees the test set during training, so its score on that set is a fair estimate of how it will do on tomorrow's unseen reviews. We pass random_state=42 so the split is the same every time you run it.

📄 train.py: train and score on the NLTK movie reviews corpus

import nltk
nltk.download("movie_reviews", quiet=True)  # one-time download

from nltk.corpus import movie_reviews
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score, classification_report

# 2,000 real movie reviews: 1,000 positive, 1,000 negative
docs = [movie_reviews.raw(fid) for fid in movie_reviews.fileids()]
labels = [1 if fid.startswith("pos") else 0 for fid in movie_reviews.fileids()]

# Hold out 25% for honest testing. random_state makes the split repeatable.
X_train, X_test, y_train, y_test = train_test_split(
    docs, labels, test_size=0.25, random_state=42, stratify=labels
)

model = Pipeline([
    ("tfidf", TfidfVectorizer(stop_words="english", max_features=5000, ngram_range=(1, 2))),
    ("clf", LogisticRegression(max_iter=1000, random_state=42)),
])
model.fit(X_train, y_train)

preds = model.predict(X_test)
print(f"Train reviews: {len(X_train)}, Test reviews: {len(X_test)}")
print(f"Test accuracy: {accuracy_score(y_test, preds):.3f}\n")
print(classification_report(y_test, preds, target_names=["negative", "positive"]))

▶ Output

Train reviews: 1500, Test reviews: 500
Test accuracy: 0.822

              precision    recall  f1-score   support

    negative       0.84      0.79      0.82       250
    positive       0.80      0.85      0.83       250

    accuracy                           0.82       500
   macro avg       0.82      0.82      0.82       500
weighted avg       0.82      0.82      0.82       500

What happened here: The model got 82.2% of the 500 unseen reviews right. For a model that just counts words and trains in a couple of seconds, that is a strong baseline. The classification_report breaks it down per class. Precision for “negative” is 0.84, meaning when the model says negative it is right 84% of the time. Recall for “negative” is 0.79, meaning it caught 79% of the actually-negative reviews. The f1-score balances those two into one number. The two classes score almost evenly, which is what you want from a balanced dataset. Your own numbers will match these exactly because we seeded the split and the model.

Step 3: Predict Brand New Reviews

A score on a test set is nice, but the real payoff is feeding the trained model a sentence you just wrote and watching it decide. Below we hand it four fresh reviews, and two of them are written as quotes from named viewers, a fan named Viraj and a disappointed viewer named Niranjan, so you can watch the model ignore the names and lock onto the sentiment words instead. predict gives the label, and predict_proba gives the confidence behind it.

📄 predict.py: classify reviews the model has never seen

# (model is the trained Pipeline from train.py)
new_reviews = [
    "An absolute masterpiece, the best film I have seen all year.",
    "Boring, predictable, and far too long. I want my money back.",
    "Viraj said the acting was brilliant and the ending was perfect.",
    "Niranjan walked out halfway through, the script was a disaster.",
]

for review in new_reviews:
    pred = model.predict([review])[0]
    proba = model.predict_proba([review])[0]
    sentiment = "Positive" if pred == 1 else "Negative"
    confidence = proba[pred]
    short = review[:45] + ("..." if len(review) > 45 else "")
    print(f'{sentiment} ({confidence:.0%})  "{short}"')

▶ Output

Positive (72%)  "An absolute masterpiece, the best film I have..."
Negative (78%)  "Boring, predictable, and far too long. I want..."
Positive (73%)  "Viraj said the acting was brilliant and the e..."
Negative (60%)  "Niranjan walked out halfway through, the scri..."

What happened here: All four reviews landed on the correct side. The clearly worded ones (“masterpiece”, “boring, predictable”) got firm confidence around 72 to 78%. Niranjan's review scored a softer 60% because “walked out halfway” is implied negativity rather than an obvious bad-word; the model is right but less sure. That confidence number is genuinely useful in production. You can route only the high-confidence predictions automatically and send the shaky ones to a human, the same way a bank auto-approves clear cases and flags the borderline ones for review.

Variation: See Why the Model Decided

A logistic regression model is not a black box. Picture a judge who, after the verdict, can point to exactly which pieces of evidence swayed the decision. Logistic regression works the same way: it learns one weight per word, and you can read those weights straight off. A big positive weight means “this word pushes toward a positive review”, a big negative weight pushes the other way. Printing the strongest words is the fastest sanity check there is: if the top positive word were “terrible”, you would know something was wired backwards.

📄 explain.py: read the words that drive each prediction

import numpy as np
# (same session as train.py, so the imports and X_train/y_train already exist)

# Retrain with single words only, so every weight below maps to exactly one word
model = Pipeline([
    ("tfidf", TfidfVectorizer(stop_words="english", max_features=5000, ngram_range=(1, 1))),
    ("clf", LogisticRegression(max_iter=1000, random_state=42)),
])
model.fit(X_train, y_train)

words = model.named_steps["tfidf"].get_feature_names_out()
coefs = model.named_steps["clf"].coef_[0]
order = np.argsort(coefs)

print("Top 8 POSITIVE words:")
for i in order[-8:][::-1]:
    print(f"  {words[i]:<12} {coefs[i]:+.2f}")

print("\nTop 8 NEGATIVE words:")
for i in order[:8]:
    print(f"  {words[i]:<12} {coefs[i]:+.2f}")

▶ Output

Top 8 POSITIVE words:
  great        +1.66
  life         +1.32
  war          +1.19
  truman       +1.16
  perfect      +1.14
  excellent    +1.13
  family       +1.03
  hilarious    +0.94

Top 8 NEGATIVE words:
  bad          -3.02
  worst        -1.76
  plot         -1.69
  boring       -1.42
  movie        -1.40
  supposed     -1.32
  script       -1.28
  reason       -1.17

What happened here: The positive list is full of words you would expect: great, perfect, excellent, hilarious. The negative list leads with bad, worst, boring. The script first retrains the pipeline with single words only (ngram_range=(1, 1)), because the bigram model from Step 2 would flood these lists with two-word features; with unigrams the output is one clean word per line.

Two entries are worth a second look. “truman” shows up positive because The Truman Show drew warm reviews in this 1990s-era corpus, and “plot” leans negative because reviewers tend to write “the plot made no sense” far more than “the plot was perfect”. That is the model learning the quirks of this particular dataset, which is exactly why you should retrain on your own domain rather than reuse someone else's weights.

Variation: Negation and N-Grams

Python sentiment analysis with single-word TF-IDF has one famous blind spot: negation. To a bag-of-words model, “not good” is just the two separate tokens “not” and “good”, and “good” pulls positive. The fix is n-grams. Setting ngram_range=(1, 2) tells TF-IDF to also keep pairs of adjacent words, so “not good” becomes its own feature, distinct from “good”. It is the difference between hearing single words shouted across a room and actually catching the two-word phrase.

📄 negation.py: do bigrams change the score?

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score

# (docs and labels loaded from movie_reviews as in train.py)
X_train, X_test, y_train, y_test = train_test_split(
    docs, labels, test_size=0.25, random_state=42, stratify=labels
)

def build(ngram_range):
    return Pipeline([
        ("tfidf", TfidfVectorizer(stop_words="english", max_features=5000,
                                  ngram_range=ngram_range)),
        ("clf", LogisticRegression(max_iter=1000, random_state=42)),
    ])

for label, rng in [("unigrams only  (1,1)", (1, 1)), ("uni + bigrams (1,2)", (1, 2))]:
    m = build(rng).fit(X_train, y_train)
    acc = accuracy_score(y_test, m.predict(X_test))
    print(f"{label}: accuracy {acc:.3f}")

# Confirm bigrams really create "not good" as a feature
vec = TfidfVectorizer(ngram_range=(1, 2))
vec.fit(["the movie was not good at all", "the movie was good"])
bigrams = [t for t in vec.get_feature_names_out() if " " in t]
print("\nSample bigram features:", bigrams)

▶ Output

unigrams only  (1,1): accuracy 0.824
uni + bigrams (1,2): accuracy 0.822

Sample bigram features: ['at all', 'good at', 'movie was', 'not good', 'the movie', 'was good', 'was not']

What happened here: Be honest about the result: on this corpus, adding bigrams did not move accuracy (0.822 versus 0.824, basically a tie). That surprises people who expect bigrams to be a magic boost. The reason is that full movie reviews are long, so a single “not good” gets drowned out by dozens of other strongly worded clues. Bigrams help much more on short text like tweets or one-line product ratings, where a single negation can flip the whole meaning. The second print proves the mechanism works: “not good” really does become its own feature. So keep ngram_range=(1, 2) as a sensible default, just do not expect miracles on long documents. When negation truly matters, reach for a context-aware transformer.

Common Mistakes

Mistake 1: Scoring the model on its training data

🚫 Wrong

model.fit(docs, labels)
preds = model.predict(docs)        # predicting on the SAME data it learned
print(accuracy_score(labels, preds))  # looks amazing, means nothing

✅ Correct

X_train, X_test, y_train, y_test = train_test_split(
    docs, labels, test_size=0.25, random_state=42, stratify=labels
)
model.fit(X_train, y_train)
preds = model.predict(X_test)      # judge on data it never saw
print(accuracy_score(y_test, preds))

Why: A model can memorize its training set and report 99% accuracy that collapses the moment it meets a real review. Always hold out a test set, or use cross-validation, so the number you report is the number you actually get in production.

Mistake 2: Fitting TF-IDF on the full dataset before splitting

🚫 Leaky

vec = TfidfVectorizer()
X = vec.fit_transform(docs)        # learns vocabulary from TEST data too
X_train, X_test, y_train, y_test = train_test_split(X, labels)

✅ Clean

# Put TF-IDF inside the Pipeline and fit only on training data
model = Pipeline([("tfidf", TfidfVectorizer()), ("clf", LogisticRegression())])
X_train, X_test, y_train, y_test = train_test_split(docs, labels)
model.fit(X_train, y_train)        # vocabulary learned from train only

Why: If TF-IDF sees the test reviews while building its vocabulary, information from the test set leaks into training and your score is quietly inflated. Wrapping the vectorizer in a Pipeline and calling fit only on the training split keeps the wall between train and test solid.

Mistake 3: Trusting accuracy on an imbalanced dataset

Why: If 95% of your reviews are positive, a lazy model that always shouts “positive” scores 95% accuracy while being useless. Watch precision, recall, and f1-score per class instead, and use stratify=labels in the split so both classes stay balanced in train and test. On our corpus the two classes are 50/50, so accuracy is fair here, but real feedback is rarely that tidy.

Best Practices

• DO wrap cleaning, TF-IDF, and the classifier in a single Pipeline so the same steps run at train and predict time
• DO seed everything (random_state=42 on the split and the model) so results are reproducible
• DO report precision, recall, and f1-score per class, not accuracy alone
• DO retrain on your own domain: a model trained on movie reviews will misread medical or financial text
• DON’T trust a model you cannot test on held-out data
• DON’T reach for BERT or a large transformer until the cheap TF-IDF baseline has shown it is not good enough

For production NLP (Natural Language Processing) work, spaCy (3.8.14 at the time of writing) and the Hugging Face transformers library give you stronger, context-aware models. But the TF-IDF baseline you built here is the honest yardstick every one of those heavier models has to beat before it earns its keep.

Conclusion

You built a complete Python sentiment analysis pipeline from scratch. You cleaned raw text, scored words with TF-IDF, trained logistic regression on 2,000 real movie reviews, and hit a solid 82% on reviews the model had never seen. Along the way you learned to read the model's own weights to see why it decided, to test bigrams honestly instead of assuming they help, and to sidestep the two classic traps: scoring on training data and trusting accuracy on an imbalanced set. That is a real, debuggable baseline you can ship today and defend in a code review.

Next you take a trained model out of the notebook and put it behind a live Application Programming Interface (API) so other apps can call it. For the full path from Python basics all the way through deploying machine learning, browse the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on Sentiment Analysis

How interviewers actually probe this topic: real scenarios, with answers you can say out loud.

Q: What does TF-IDF actually measure, and why is it better than plain word counts for sentiment?

TF-IDF weighs each word by term frequency (how often it appears in one review) times inverse document frequency (how rare it is across all reviews). Plain counts let common filler words like “the” dominate every vector. TF-IDF pushes those everyday words down and lifts rare, distinctive words like “brilliant” or “awful” up, which are exactly the words that carry sentiment. That reweighting is why a simple linear model on TF-IDF is such a strong baseline.

Q: Why should the TfidfVectorizer be fitted only on the training split and never on the full dataset?

Fitting the vectorizer learns the vocabulary and the IDF weights. If it sees the test reviews during that step, information about the test set leaks into your features and your reported score is quietly inflated. The clean fix is to put the vectorizer inside a Pipeline and call fit only on the training data, so the vocabulary is built from train alone and the test set stays a fair proxy for unseen reviews.

Q: Your notebook reports 97% accuracy, but in production the model labels almost every review positive. What do you check first?

First check for class imbalance in the data you evaluated on: if the test set was mostly positive, a model that always predicts positive scores high on accuracy while being useless. Look at precision, recall, and f1-score per class, not the single accuracy number. Then check for leakage, such as fitting TF-IDF before the split or accidentally scoring on training rows. A per-class report plus a proper held-out split usually exposes which of the two it is.

Q: A user submits “the plot was not good at all” and your unigram model calls it positive. How do you fix it?

A bag-of-words model treats “not” and “good” as separate tokens, and “good” pulls positive. Set ngram_range=(1, 2) so TF-IDF also keeps adjacent pairs, turning “not good” into its own feature that the classifier can learn as negative. Bigrams help most on short text like this, where one negation flips the whole meaning. If negation keeps hurting you on longer, trickier text, move up to a context-aware transformer such as BERT.

Q: How would you inspect what your logistic regression model learned?

Logistic regression stores one coefficient per feature, so you can line up coef_[0] against get_feature_names_out() and sort. The largest positive weights are the words pushing toward “positive” and the most negative weights push the other way. This is a fast sanity check: if a word like “terrible” showed up with a strong positive weight, you would know your labels were flipped or your pipeline was wired backwards.

Q: When would you drop TF-IDF plus logistic regression and reach for a transformer model?

Reach for a transformer only after the cheap baseline has clearly hit its ceiling on your data. Transformers win when meaning depends on context and word order, such as sarcasm, subtle negation, or long-range references that a bag-of-words model cannot see. The trade-off is cost: they need far more compute and are harder to debug. The professional habit is to ship the TF-IDF baseline first, measure it honestly, and let that number decide whether the heavier model is worth it.

Series: Python + AI/ML Cookbook, Part 5: Machine Learning

Want more? the official Python documentation documents everything this post could not fit.

Related Posts

Previous: ML: NLP Basics, Tokenization and TF-IDF

Next: Your First Kaggle Competition: An ML Project That Counts

Series Home: Python + AI/ML Tutorial Series