Python logistic regression is the everyday workhorse for classification: the sigmoid function, binary and multi-class problems, reading the output as a real probability, decision boundaries, and knowing when to pick it over other classifiers.

“If you can’t get logistic regression to work, don’t bother with neural networks.”

Andrew Ng, Stanford CS229

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0 | Difficulty: Intermediate | Reading Time: 14 minutes

Linear regression predicts a number. But a lot of real questions are yes or no. Is this email spam? Will this customer cancel? Does this patient have the disease? You could try linear regression and just call anything above 0.5 a “yes”, but its output is not capped between 0 and 1. It happily spits out a “probability” of -0.3 or 1.7, and a 170% chance of anything is nonsense.

Logistic regression fixes that with one extra step. It takes the linear output and runs it through a sigmoid function, which squashes any number, big or small, into the range 0 to 1. Now the result reads like a real probability: “there is a 78% chance this email is spam.” Think of the sigmoid as a volume knob that can never go below zero or above the max, no matter how hard you turn it. You keep that probability, then pick a cutoff (usually 0.5) to make the final yes or no call.

Here is why that probability matters in real life. Rahul, an engineer on a subscription product, needed to spot which customers were about to cancel. Logistic regression hit 87% accuracy, but the real win was the probability it handed back for each customer. Someone at 90% churn risk got an instant call from the retention team. Someone at 55% got a friendly email with a discount. The number, not just the yes or no, decided what action to take. A plain “this customer will cancel” label could never do that.

Prerequisites

The Sigmoid Function: Squashing to Probability

Read the diagram top to bottom. Your features get combined into one number (that is the linear part, z). The sigmoid bends that number into a probability between 0 and 1. The 0.5 line then decides the class. One quick note on the name that trips up everyone: logistic regression is a classifier, not a regressor. The word “regression” only points at the straight-line equation hiding inside the sigmoid. The big payoff of this whole pipeline is that you get a probability, not just a label, so the model tells you how sure it is, not only what it picked.

📄 sigmoid.py: understanding the sigmoid function

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# The sigmoid maps any real number to (0, 1)
z_values = np.array([-10, -5, -2, -1, 0, 1, 2, 5, 10])
probs = sigmoid(z_values)

print(f"{'z':>6} | {'sigmoid(z)':>10} | {'Interpretation'}")
print("-" * 50)
for z, p in zip(z_values, probs):
    interp = "Very unlikely" if p < 0.2 else "Unlikely" if p < 0.4 else \
             "Uncertain" if p < 0.6 else "Likely" if p < 0.8 else "Very likely"
    print(f"{z:>6.0f} | {p:>10.4f} | {interp}")

print(f"\nKey properties:")
print(f"  sigmoid(0) = {sigmoid(0):.1f} (the decision boundary)")
print(f"  sigmoid(-∞) → 0, sigmoid(+∞) → 1")
print(f"  Output is always between 0 and 1 (valid probability)")

▶ Output

     z | sigmoid(z) | Interpretation
--------------------------------------------------
   -10 |     0.0000 | Very unlikely
    -5 |     0.0067 | Very unlikely
    -2 |     0.1192 | Very unlikely
    -1 |     0.2689 | Unlikely
     0 |     0.5000 | Uncertain
     1 |     0.7311 | Likely
     2 |     0.8808 | Very likely
     5 |     0.9933 | Very likely
    10 |     1.0000 | Very likely

Key properties:
  sigmoid(0) = 0.5 (the decision boundary)
  sigmoid(-∞) → 0, sigmoid(+∞) → 1
  Output is always between 0 and 1 (valid probability)

What happened here: Look at the shape of that table. A very negative z (like -10) gives a probability near 0, a very positive z (like 10) gives a probability near 1, and z = 0 lands exactly on 0.5. That 0.5 point is the fence. The curve is an S, steep in the middle and flat at both ends, so once you are confident the model stops changing its mind much. The formula itself is tiny: 1 / (1 + e^-z). That one line is the whole reason logistic regression can turn any score into a clean, well-behaved probability.

Python Logistic Regression for Spam Detection

Time to use it on a real yes-or-no job: is an email spam? We will hand the model four clues about each email, the word count, the number of links, how many exclamation marks it has, and how much of it is shouting in uppercase. Python logistic regression will learn how much each clue matters and give back a spam probability for every email.

📄 binary_classification.py: logistic regression for spam detection

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
from sklearn.preprocessing import StandardScaler

rng = np.random.default_rng(42)
n = 500

# Features: word_count, link_count, excl_marks, uppercase_ratio
word_count = rng.integers(10, 200, n).astype(float)
link_count = rng.integers(0, 10, n).astype(float)
excl_marks = rng.integers(0, 15, n).astype(float)
upper_ratio = rng.uniform(0, 0.5, n)

# Spam if: many links + many exclamation marks + high uppercase
spam_score = 0.5 * link_count + 0.3 * excl_marks + 3 * upper_ratio - 0.01 * word_count
# Split at the median so we get a roughly balanced 50/50 dataset
threshold = np.median(spam_score)
is_spam = (spam_score + rng.normal(0, 0.7, n) > threshold).astype(int)

X = np.column_stack([word_count, link_count, excl_marks, upper_ratio])
feature_names = ["word_count", "link_count", "excl_marks", "uppercase_ratio"]

X_train, X_test, y_train, y_test = train_test_split(X, is_spam, test_size=0.2,
                                                       random_state=42, stratify=is_spam)

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

model = LogisticRegression(random_state=42)
model.fit(X_train_s, y_train)

# Coefficients tell you which features matter
print("Feature Coefficients (scaled):")
for name, coef in zip(feature_names, model.coef_[0]):
    direction = "→ spam" if coef > 0 else "→ not spam"
    print(f"  {name:<16}: {coef:>7.3f} {direction}")

# Predictions with probabilities
y_pred = model.predict(X_test_s)
y_proba = model.predict_proba(X_test_s)

print(f"\nAccuracy: {accuracy_score(y_test, y_pred):.1%}")
print(f"\n{classification_report(y_test, y_pred, target_names=['Not Spam', 'Spam'])}")

# Sample predictions with probabilities
for i in [0, 5, 10]:
    prob = y_proba[i][1]
    pred = "Spam" if prob > 0.5 else "Not Spam"
    print(f"  Email {i}: P(spam)={prob:.2%}, predicted={pred}")

▶ Output

Feature Coefficients (scaled):
  word_count      :  -1.082 → not spam
  link_count      :   3.188 → spam
  excl_marks      :   2.578 → spam
  uppercase_ratio :   1.019 → spam

Accuracy: 87.0%

              precision    recall  f1-score   support

    Not Spam       0.88      0.87      0.87        52
        Spam       0.86      0.88      0.87        48

    accuracy                           0.87       100
   macro avg       0.87      0.87      0.87       100
weighted avg       0.87      0.87      0.87       100

  Email 0: P(spam)=79.71%, predicted=Spam
  Email 5: P(spam)=77.30%, predicted=Spam
  Email 10: P(spam)=21.72%, predicted=Not Spam

What happened here: The model figured out the pattern on its own. More links, more exclamation marks, and more uppercase push the spam probability up (positive coefficients), while longer emails lean toward not-spam (the negative word_count coefficient). The biggest number wins: link_count at 3.188 is the loudest spam signal here. The nice thing is you can read these coefficients out loud to a non-technical boss and explain exactly why an email got flagged. No black box.

And look at the last three lines. The model is 87% accurate overall, but the probabilities are the real gift. Email 0 sits at 79.71% (confident spam), email 5 at 77.30% (also spam but a touch less sure), and email 10 at 21.72% (clearly fine). It is like a smoke detector that tells you “thick smoke” versus “faint whiff” instead of just beeping. You can send the 80%+ emails straight to the junk folder and quarantine the borderline ones for a human to glance at. A plain yes or no throws all of that nuance away.

Multi-class Classification

Spam was two choices. What if you have three or more? Sorting an iris flower into setosa, versicolor, or virginica, say. Python logistic regression handles this with softmax, which is just the sigmoid’s bigger sibling. Instead of one probability, softmax hands back a probability for every class, and they all add up to 100%. Picture a pie chart: the model slices the whole pie among the classes, and the biggest slice wins. Good news for the code: in scikit-learn 1.9.0 you do nothing special. Give it three or more labels and it switches to softmax automatically.

📄 multiclass.py: classifying into 3 or more categories

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42, stratify=iris.target
)

# With 3+ classes, scikit-learn 1.9 fits a softmax (multinomial)
# model automatically: one model that outputs a probability per class.
# (The old multi_class= argument has been removed in current scikit-learn,
#  so you no longer pass it.)
model = LogisticRegression(max_iter=200, random_state=42)
model.fit(X_train, y_train)

print(f"Accuracy: {model.score(X_test, y_test):.1%}")
print(f"Classes: {iris.target_names.tolist()}")

# Softmax gives probability for each class
proba = model.predict_proba(X_test[:3])
for i, (pred, prob) in enumerate(zip(model.predict(X_test[:3]), proba)):
    class_name = iris.target_names[pred]
    probs_str = ", ".join([f"{iris.target_names[j]}={p:.0%}" for j, p in enumerate(prob)])
    print(f"\nSample {i}: predicted={class_name}")
    print(f"  Probabilities: {probs_str}")

▶ Output

Accuracy: 96.7%
Classes: ['setosa', 'versicolor', 'virginica']

Sample 0: predicted=setosa
  Probabilities: setosa=99%, versicolor=1%, virginica=0%

Sample 1: predicted=virginica
  Probabilities: setosa=0%, versicolor=39%, virginica=61%

Sample 2: predicted=versicolor
  Probabilities: setosa=19%, versicolor=81%, virginica=0%

What happened here: One model, three probabilities per flower, and they sum to 100% every time. Sample 0 is an easy call at 99% setosa. Sample 2 leans versicolor at 81%. The interesting one is sample 1: 39% versicolor versus 61% virginica. The model still picks virginica (the bigger slice), but it is openly telling you “I am not very sure here.” That honesty is the whole point. With softmax you do not just get a label, you get a confidence level you can act on, like flagging the close calls for a second look.

Common Mistakes

Think of a weather app in a desert that predicts “no rain” every single day. It is right about 360 days a year, so it brags about 98% accuracy, yet it is worthless on the one day a flash flood is coming. Accuracy alone hides the failure that actually matters. Classifiers fall into the same trap. Imagine 95% of your emails are not spam. A lazy model that labels everything “not spam” scores 95% accuracy and catches exactly zero spam. The grade looks great, the model is useless. So always read precision, recall, and F1, not just accuracy.

❌ Mistake: Using accuracy alone on imbalanced data

# If 95% of emails are not spam, a model that always predicts "not spam"
# gets 95% accuracy, but catches zero spam!
# Always check precision, recall, and F1 for classification:
# - Precision: of the emails we LABELLED spam, how many really were spam?
# - Recall:    of the emails that ARE spam, how many did we catch?
# - F1:        a single score that balances precision and recall

print("Accuracy alone is misleading on imbalanced datasets.")
print("Use classification_report() to see precision, recall, and F1.")
print("Catching every positive case matters? Watch RECALL (e.g. screening for a disease).")
print("A false alarm is very costly? Watch PRECISION (e.g. blocking a legit email).")

▶ Output

Accuracy alone is misleading on imbalanced datasets.
Use classification_report() to see precision, recall, and F1.
Catching every positive case matters? Watch RECALL (e.g. screening for a disease).
A false alarm is very costly? Watch PRECISION (e.g. blocking a legit email).

Practice Exercises

1. Exercise 1: Take the spam example above and lower the decision cutoff from 0.5 to 0.3. Re-run the classification report. Does recall on the Spam class go up? Does precision drop? Explain the trade-off in one sentence.
2. Exercise 2: Train the model on just two features (link_count and excl_marks) and plot the decision boundary as a 2D scatter with the dividing line. Where does the line fall?
3. Exercise 3: Implement the sigmoid and a simple gradient-descent training loop from scratch with NumPy, then check that your learned weights roughly match scikit-learn’s on the same toy data.

Conclusion

You now know what makes Python logistic regression tick: the sigmoid function that squashes any score into a clean 0 to 1 probability, binary classification with readable coefficients, multi-class problems handled by softmax, and why you should never judge a classifier on accuracy alone. The theme running through all of it is the same: this model hands you a probability, not just a label, so it tells you how sure it is, and that number is what lets you act intelligently.

Next up is model evaluation metrics, where we dig into accuracy, precision, recall, and F1 so you can judge a classifier properly instead of trusting one number. For the full roadmap from Python basics to machine learning, visit the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on Logistic Regression

Try each one aloud first. The phrasing you produce under mild pressure is what interviews measure.

Q: What does the sigmoid function do, and why is it the heart of logistic regression?

The sigmoid, 1 / (1 + e^-z), takes the linear score z (the weighted sum of the features plus a bias) and squashes it into the range 0 to 1. That output reads as a probability, so a very negative z maps near 0, a very positive z maps near 1, and z = 0 lands exactly on 0.5. Without it you would have a plain linear regression that can output values below 0 or above 1, which cannot be interpreted as a probability.

Q: What is the difference between predict and predict_proba in scikit-learn?

predict returns the final class label after applying the 0.5 threshold, so you get a 0 or 1 (or a class name for multi-class). predict_proba returns the raw probabilities for every class, for example [0.22, 0.78]. Use predict_proba when you need to rank cases by confidence, set a custom threshold, or feed the probability into a downstream decision, which is usually the real value of logistic regression.

Q: Why should you scale features before fitting a logistic regression model?

Logistic regression uses regularization by default (the C parameter), and regularization penalizes large coefficients. If one feature is measured in thousands and another in fractions, the penalty hits them unevenly and the optimizer also converges more slowly. Standardizing features with StandardScaler puts them on a comparable scale, so the coefficients become fairly comparable in size and training is faster and more stable.

Q: How do you interpret the coefficients of a fitted logistic regression?

The sign tells you direction: a positive coefficient pushes the probability of the positive class up as that feature grows, a negative one pushes it down. On scaled features, the magnitude tells you relative importance, so the largest absolute coefficient is the strongest signal. More precisely, each coefficient is the change in the log-odds per unit change in that feature, and exponentiating it gives an odds ratio.

Q: Your fraud classifier reports 97% accuracy, but the team says it never actually catches fraud. What do you check first?

This is the classic imbalanced-data trap: if only 3% of transactions are fraud, a model that always predicts “not fraud” scores 97% accuracy while catching zero cases. Stop looking at accuracy and pull the classification report to read precision, recall, and F1 for the fraud class specifically. Then address the imbalance with class_weight=”balanced”, resampling, or by lowering the decision threshold to raise recall.

Q: You get a ConvergenceWarning saying the solver failed to converge. What are your options?

First, scale your features, since unscaled data is the most common cause of slow convergence. If it still warns, raise max_iter (for example from the default 100 to 1000) to give the optimizer more steps. You can also try a different solver such as saga or lbfgs, and double-check that no feature is constant or perfectly collinear with another, which can stall the fit.

Q: When would you lower the decision threshold below 0.5, and what is the trade-off?

Lower the threshold when missing a positive case is far more costly than a false alarm, such as screening for a disease or flagging fraud. Dropping the cutoff from 0.5 to, say, 0.3 catches more true positives, so recall goes up, but you also flag more borderline cases that turn out negative, so precision drops. You pick the threshold based on which error hurts more in your specific problem.

Series: Python + AI/ML Cookbook, Part 5: Machine Learning

Want more? scikit-learn documentation documents everything this post could not fit.

Related Posts

Previous: ML: Polynomial & Regularization (Ridge, Lasso)

Next: ML: Model Evaluation Metrics (Accuracy, Precision, Recall, F1)

Series Home: Python + AI/ML Tutorial Series