You trained a model in a notebook and the accuracy looks great. Now your teammate asks, “Cool, how do I actually use it?” That question is where most models quietly die. This guide walks the Python ML model serving path from a saved notebook model to a live prediction API (Application Programming Interface): serialize the model with joblib, serve it with FastAPI, validate the incoming data with Pydantic, and version it so you can roll back when something breaks.

“Most machine learning models die in notebooks. The ones that survive make it to production because someone figured out how to serialize, serve, and monitor them.”

Chip Huyen, Designing ML Systems

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0, FastAPI 0.138.0, Pydantic 2.13.4 | Difficulty: Advanced | Reading Time: 14 minutes

Training a model is maybe 20 percent of a real Machine Learning (ML) project. The other 80 percent is getting it somewhere it can make predictions on live data. Think of a trained model like a cake you baked at home: tasting it in your kitchen is easy, but selling slices to customers needs a counter, a till, and someone checking the orders. The model is the cake. Serialization is the box you carry it in. The API is the counter. Pydantic is the person checking that each order makes sense before it reaches the kitchen.

This post covers the first, most important version of that counter for your Python ML model: train in a notebook, save with joblib, serve with FastAPI, and validate inputs with Pydantic. This is not “ML at massive scale” with autoscaling and GPUs (that is the ML deployment at scale tutorial). This is “get your first model answering real requests without falling over.”

Prerequisites

The Deployment Flow

Before any code, here is the shape of what we are building. Read it top to bottom: it is a straight line, not a maze.

Here is what each box means in plain words. You train the model in a notebook like usual. You serialize it, which just means freezing the trained object into a file on disk with joblib. Your FastAPI endpoint is the web address that accepts requests. When the service boots, it loads the model on startup one time and keeps it in memory. After that, every request flows the same way: the API receives JSON (JavaScript Object Notation) input, runs it through the model, and returns the prediction as JSON.

The Python ML model and the serving code stay separate, so a data scientist can retrain and swap the file without anyone touching the API code. That separation is the whole point.

Install and Verify

The Python ML deployment stack here is three libraries: scikit-learn to build the model, FastAPI to serve it, and uvicorn to actually run the server. Pydantic comes bundled with FastAPI, and joblib ships inside scikit-learn, so you do not install those separately.

📄 Install the deployment stack

pip install scikit-learn fastapi "uvicorn[standard]"

To confirm the versions you actually have, print them. Pin these exact numbers later in your requirements.txt so production runs the same code your notebook did.

📄 check_versions.py

import sklearn, fastapi, pydantic, joblib
print("scikit-learn:", sklearn.__version__)
print("fastapi:", fastapi.__version__)
print("pydantic:", pydantic.__version__)
print("joblib:", joblib.__version__)

▶ Output

scikit-learn: 1.9.0
fastapi: 0.138.0
pydantic: 2.13.4
joblib: 1.5.3

These are the versions tested for this post. Your numbers may be newer, which is fine. The patterns below do not change, but always pin whatever you actually use.

Step 1: Serialize the Model

Serialization is a long word for a simple idea: turn your trained model object into bytes you can save to a file and load back later. Joblib is the go-to for scikit-learn models because it stores big NumPy arrays efficiently. Pickle from the standard library works too, just slower on large arrays. One safety rule worth tattooing on your brain: never load a pickle or joblib file you did not create, because opening it can run arbitrary code on your machine. Treat a model file like a USB stick a stranger handed you.

📄 save_model.py: train and serialize a pipeline

import joblib
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
import os

# Train a pipeline on a small toy dataset (seeded, so results are reproducible)
X, y = make_classification(n_samples=1000, n_features=10, random_state=42)

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", RandomForestClassifier(n_estimators=100, random_state=42)),
])

scores = cross_val_score(pipeline, X, y, cv=5, scoring="accuracy")
print(f"CV Accuracy: {scores.mean():.3f}")

pipeline.fit(X, y)

# Save the WHOLE pipeline (scaler + model) with joblib
model_path = "model_v1.joblib"
joblib.dump(pipeline, model_path)
file_size = os.path.getsize(model_path)
print(f"Model saved to {model_path} ({file_size / 1024:.0f} KB)")

# Load it back and check the predictions are identical
loaded = joblib.load(model_path)
sample = X[:3]
original_preds = pipeline.predict(sample)
loaded_preds = loaded.predict(sample)
print(f"Predictions match: {np.array_equal(original_preds, loaded_preds)}")

# Cleanup
os.remove(model_path)
print("Model file cleaned up.")

▶ Output

CV Accuracy: 0.904
Model saved to model_v1.joblib (1193 KB)
Predictions match: True
Model file cleaned up.

What happened here: The whole pipeline, scaler and model together, got frozen into one file (about 1.2 MB for this 100-tree forest). Loading it back gives identical predictions, which is the check that proves the save worked. The important detail is that we saved the entire pipeline, not just the model. The scaler remembers the mean and standard deviation it learned during training, and those numbers are baked into the file. If you save only the model and forget the scaler, predictions in production come out wrong because the incoming data never got scaled the same way. It is like saving a recipe but forgetting to write down that you doubled the salt.

Step 2: Serve with FastAPI

Now we wrap that saved Python ML model in an API. FastAPI gives you two things that matter here: it loads the model once and reuses it, and it uses Pydantic to check every incoming request before your model ever sees it. If someone sends 3 numbers when the model needs 10, Pydantic rejects the request with a clear error instead of letting your model crash on a bad shape.

In real life you start this app with uvicorn app:app --reload and hit it over HTTP (HyperText Transfer Protocol). To prove the code actually works inside this post, we drive it with FastAPI’s built-in TestClient, which sends real requests through the app and gives back real responses. No separate server needed, and the output below is exactly what the endpoint returns.

📄 app.py: a real FastAPI prediction endpoint

import joblib
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification

from fastapi import FastAPI
from pydantic import BaseModel, Field
from fastapi.testclient import TestClient

# One-time setup: train and save a model so the app has something to load.
X, y = make_classification(n_samples=1000, n_features=10, random_state=42)
pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", RandomForestClassifier(n_estimators=100, random_state=42)),
])
pipeline.fit(X, y)
joblib.dump(pipeline, "model_v1.joblib")

# ---- the actual app ----
app = FastAPI(title="ML Prediction API")

# Load the model ONCE at startup, not on every request.
model = joblib.load("model_v1.joblib")

class PredictionInput(BaseModel):
    # Exactly 10 numeric features. Pydantic enforces the length for us.
    features: list[float] = Field(..., min_length=10, max_length=10)

class PredictionOutput(BaseModel):
    prediction: int
    probability: float
    model_version: str = "v1"

@app.post("/predict", response_model=PredictionOutput)
def predict(data: PredictionInput):
    row = np.array(data.features).reshape(1, -1)
    pred = model.predict(row)[0]
    proba = model.predict_proba(row)[0].max()
    return PredictionOutput(
        prediction=int(pred),
        probability=round(float(proba), 2),
    )

# ---- drive the app with TestClient: these are real HTTP responses ----
client = TestClient(app)

good = {"features": [0.5, -1.2, 0.3, 1.1, -0.7, 0.2, -0.4, 0.9, -1.5, 0.6]}
r1 = client.post("/predict", json=good)
print("Valid request  ->", r1.status_code, r1.json())

bad = {"features": [0.5, -1.2, 0.3]}   # only 3 features instead of 10
r2 = client.post("/predict", json=bad)
print("Bad request    ->", r2.status_code, "(Pydantic rejected it)")

▶ Output

Valid request  -> 200 {'prediction': 1, 'probability': 0.55, 'model_version': 'v1'}
Bad request    -> 422 (Pydantic rejected it)

What happened here: The valid request returned HTTP 200 with a clean JSON answer: class 1, the model’s confidence (0.55), and the version tag. The bad request, only 3 features, never reached the model at all. Pydantic saw that features failed the min_length=10 rule and sent back HTTP 422 (Unprocessable Entity) on its own. That is the quiet strength of this setup: bad input is bounced at the door, like a bouncer checking IDs before anyone gets into the club, so your model only ever runs on data shaped the way it expects.

Notice the model is loaded exactly once, right after app = FastAPI(...), not inside predict(). Loading it per request would re-read 1.2 MB from disk on every single call and make your API painfully slow.

Versioning Your Model

Think of your model files like leftovers in the fridge: a container with no date and no label is a gamble you will lose. Python ML models are not “save once and forget.” You will retrain as new data arrives, and one day a new model will be worse than the old one in some sneaky way. When that happens you want to roll back in seconds, which means every model needs a version you can point to. The cheapest version system that actually works: put the version in the filename and keep a tiny notes file next to it.

📄 A simple, honest versioning scheme

models/
  model_v1.joblib       # the frozen pipeline
  model_v1.json         # metadata: trained_on, accuracy, sklearn version, feature names
  model_v2.joblib
  model_v2.json

The metadata file is the part people skip and regret. Six months from now, “which dataset trained model_v1 and why is its accuracy 0.90?” is a question you cannot answer from a .joblib file alone. Store the training date, the metric, the exact library versions, and the feature order. For a serious project, graduate to a real model registry like MLflow, which does all of this plus a UI. But a filename and a JSON file will carry you a long way, and it beats having no versioning at all.

Common Mistakes

Mistake 1: Forgetting to pin the scikit-learn version

A model saved with one scikit-learn version can refuse to load, or load with a scary warning, in a different version. The internal layout of the saved objects changes between releases. The fix is boring but bulletproof: record the version that trained the model and pin it exactly in production.

✅ Record and pin the version

import sklearn
print(f"scikit-learn version: {sklearn.__version__}")
print("Save this version next to your model file.")
print("In production, pin the exact version in requirements.txt:")
print("  scikit-learn==1.9.0  # exact pin, not >=1.9")

▶ Output

scikit-learn version: 1.9.0
Save this version next to your model file.
In production, pin the exact version in requirements.txt:
  scikit-learn==1.9.0  # exact pin, not >=1.9

Why: scikit-learn>=1.9 in a requirements file is a time bomb. A future minor release can change how the model deserializes, and your API starts erroring at 2 a.m. ==1.9.0 freezes it. When you retrain on a newer version, you bump the pin on purpose, after testing, not by accident.

Mistake 2: Loading the model inside the request handler

Putting joblib.load(...) inside the predict() function feels harmless until you measure it. Every request then re-reads the whole model file from disk, which can be tens or hundreds of milliseconds of pure waste per call. Load it once at module level, the way the app above does, and reuse the object across every request.

Conclusion

You just took a model out of the notebook and put it behind a real API. You serialized the whole pipeline with joblib so the scaler travels with the model, served it with FastAPI so the file loads once and answers many requests, let Pydantic bounce bad input at the door before the model ever sees it, and added a plain filename-plus-metadata scheme so you can roll back the day a new model misbehaves. That is the honest minimum version of production Python ML serving, and it is enough to ship something people can actually call.

Next, learn the practices that keep this model trustworthy once real users depend on it in the ML best practices tutorial, and when traffic grows, move on to the ML deployment at scale tutorial. For the full path from Python basics through AI/ML, browse the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on ML Model Serving

These come from real screens and onsites. Practice answering before you read each answer.

Q: Why is it better to save the entire scikit-learn Pipeline instead of just the trained model?

The Pipeline bundles the preprocessing (like the StandardScaler) together with the estimator, and the scaler holds the mean and standard deviation it learned during training. If you save only the model, production data never gets scaled the same way, so predictions come out wrong even though nothing errors. Saving the whole pipeline guarantees the exact same transform is applied at inference time as during training.

Q: Why load the model at module level instead of inside the request handler?

Loading inside the handler re-reads the whole model file from disk on every request, which adds tens or hundreds of milliseconds of pure waste per call. Loading once at startup keeps the deserialized object in memory and reuses it across all requests. This is the single biggest easy win for serving latency.

Q: What role does Pydantic play in a FastAPI prediction endpoint?

Pydantic validates the request body against your declared schema before the function runs, so malformed input (wrong number of features, wrong types) is rejected with a 422 before the model is ever called. It turns a potential crash deep inside NumPy into a clear, structured error message that names the failing field. In the example, Field(..., min_length=10, max_length=10) enforces exactly 10 features for free.

Q: Why pin the exact scikit-learn version in production rather than using a minimum like >=1.9?

A model pickled with one scikit-learn version can fail to load, or load with a warning, under a different version because the internal object layout changes between releases. A floating pin such as >=1.9 lets a future minor release slip in silently and break deserialization at the worst possible time. Pinning ==1.9.0 freezes the runtime to the version that trained the model; you bump it deliberately after retraining and testing.

Q: Your prediction API returns HTTP 422 for every request from a new client, but the same payloads work in your tests. What do you check first?

A 422 means Pydantic rejected the body before the model ran, so the problem is the request shape, not the model. Read the 422 response body: it names the exact field and rule that failed. Usually the client is sending the wrong number of features, wrong JSON types (strings instead of floats), or the wrong key name (not features). Compare their payload byte for byte against your PredictionInput schema.

Q: You deploy a freshly retrained model and prediction accuracy quietly drops in production, though nothing throws an error. How do you recover and diagnose it?

First recover: because you versioned the model in the filename, point the service back at the previous model_v1.joblib and restart, so you are rolling back in seconds rather than debugging live. Then diagnose using the metadata file for each version: compare training date, dataset, library versions, and feature order between the good and bad models. A silent accuracy drop with no error usually traces to data drift, a changed feature order, or a scaler mismatch, and the side-by-side metadata is what surfaces it.

Series: Python + AI/ML Cookbook. Part 5: Machine Learning

Go deeper: when you outgrow this post, the official Python documentation is the next stop.

Related Posts

Previous: Your First Kaggle Competition: An ML Project That Counts

Next: Python: Machine Learning Operations (MLOps) with MLflow, Experiment Tracking and Model Versioning

Series Home: Python + AI/ML Tutorial Series