You trained a model last Tuesday and it scored 94%. Today your boss asks which data you used, which settings, and whether you can rebuild that exact model. If the answer lives in a file named model_final_v2_REAL.pkl, you already feel the pain that Python MLOps solves. This guide covers experiment tracking and model versioning with MLflow, the most widely used open-source tool for the job.

“Only a small fraction of real-world ML systems is composed of the ML code. The required surrounding infrastructure is vast and complex.”

D. Sculley et al., Hidden Technical Debt in Machine Learning Systems (NeurIPS 2015)

Last Updated: July 2026 | Tested on: Python 3.14.6, mlflow 3.14.0, scikit-learn 1.9.0 | Difficulty: Advanced | Reading Time: 12 minutes

MLOps is the set of practices for deploying, monitoring, and maintaining machine learning models in production. Think of it like running a restaurant kitchen. Cooking one great dish at home is easy. Serving that same dish, at the same quality, to 500 customers every night, with new ingredients arriving daily, is a whole different job. Training a model is maybe 20% of the work. The other 80% is versioning your data and models, tracking experiments, automating retraining, and catching model drift before your users notice. MLflow is the most widely adopted open-source platform for managing this whole lifecycle.

This post covers the parts of MLflow you reach for daily: experiment tracking, model versioning, and the model registry. By the end you will manage machine learning (ML) experiments in a calm, organized way instead of drowning in folders named model_v2_final_FINAL_actually_final. Everything here was run on Python 3.14.6 with MLflow 3.14.0 (latest stable at the time of writing), so the output you see is the real output, not a screenshot from someone else’s machine.

The MLOps Lifecycle

Think of it like a car factory that never really shuts down: raw materials roll in one end, finished cars roll out the other, and inspectors keep sending feedback upstream so the next batch comes out better. Python MLOps is that same never-ending loop for models. The diagram shows the MLOps loop, and MLflow plugs into the green middle layer of it. MLflow ships four building blocks: Tracking (logging parameters, metrics, and artifacts for every run), Projects (packaging ML code so it runs the same way anywhere), Models (a standard format for saving and serving a model), and the Model Registry (versioning and promoting models toward production).

Together they give every model a paper trail, so you can always answer “where did this model come from?”. The two blocks you touch every day are Tracking and the Model Registry, so those are the two we will actually code below.

Install and Verify

Install MLflow and scikit-learn with pinned versions so your results match the post, then check the version from the command line.

📄 terminal: install and check the version

pip install mlflow==3.14.0 scikit-learn==1.9.0
mlflow --version

▶ Output

mlflow, version 3.14.0

If you see mlflow, version 3.14.0, you are ready. If mlflow is “not found”, your scripts directory is not on PATH, so use python -m mlflow --version instead. One quick note on naming: in MLflow 3, the old model “stages” (Staging, Production) are deprecated and are being replaced by named aliases. We use aliases throughout this post because that is the current, supported way.

Experiment Tracking: Every Run, Every Metric, Every Parameter

Experiment tracking is just a lab notebook for your models. Every time you train one, MLflow writes down the settings you used, the scores you got, and a copy of the model itself. Picture a chef testing recipe versions: same dish, slightly different salt, oven time, or flour, and a little card pinned next to each tray saying what changed and how it tasted. That card is exactly what mlflow.start_run() creates for you, automatically, for every run. It is the highest-value habit in Python MLOps: a few extra lines per training script, and no experiment is ever lost again.

In the script below, say a data scientist named Rahul trains three different classifiers on the iris dataset and logs each one. Notice the name="model" argument on log_model: in MLflow 3 that keyword is the current spelling (older code passed it as a bare positional artifact_path).

📄 experiment_tracking.py: MLflow experiment tracking

import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, f1_score

# Rahul sets up MLflow experiment tracking
mlflow.set_tracking_uri("sqlite:///mlflow.db")  # Local SQLite for experiments
mlflow.set_experiment("iris-classification")

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Run multiple experiments
experiments = [
    {"name": "RandomForest-100", "model": RandomForestClassifier(n_estimators=100, random_state=42)},
    {"name": "RandomForest-200", "model": RandomForestClassifier(n_estimators=200, max_depth=5, random_state=42)},
    {"name": "GradientBoosting", "model": GradientBoostingClassifier(n_estimators=100, random_state=42)},
]

for exp in experiments:
    with mlflow.start_run(run_name=exp["name"]):
        model = exp["model"]
        model.fit(X_train, y_train)
        preds = model.predict(X_test)

        accuracy = accuracy_score(y_test, preds)
        f1 = f1_score(y_test, preds, average="weighted")

        # Log parameters, metrics, and model
        mlflow.log_params(model.get_params())
        mlflow.log_metrics({"accuracy": accuracy, "f1_score": f1})
        mlflow.sklearn.log_model(model, name="model")

        print(f"  {exp['name']:>25}: accuracy={accuracy:.4f}, f1={f1:.4f}")

print(f"\nView experiments: mlflow ui --backend-store-uri sqlite:///mlflow.db")

▶ Output

           RandomForest-100: accuracy=1.0000, f1=1.0000
           RandomForest-200: accuracy=1.0000, f1=1.0000
           GradientBoosting: accuracy=1.0000, f1=1.0000

View experiments: mlflow ui --backend-store-uri sqlite:///mlflow.db

What happened here: Each mlflow.start_run() opens a tracked run that records the parameters (the model’s settings), the metrics (accuracy and F1), and a saved copy of the trained model. MLflow tucks all of it into a local SQLite file called mlflow.db. Run mlflow ui --backend-store-uri sqlite:///mlflow.db and a web dashboard opens at http://localhost:5000 where you can line up the runs side by side, sort by any metric, and download a model.

That is how a team stops asking “wait, which notebook had the good model?”. All three models score a perfect 1.0000 here because iris is a tiny, very easy dataset with only 150 rows, so do not read too much into the perfect score. On real data you will see the runs actually differ, and that is the moment tracking earns its keep.

Model Registry: Version Control for Models

Tracking saves every run. The registry is where you pick the winners and name them. Think of it like the contacts app on your phone: the model files are the actual phone numbers, and an alias like staging or production is a label such as “Mom” or “Work” that points at one of them. When you swap in a newer, better model, you just move the production label to the new version. Your serving code keeps asking for “production” and never has to care which exact number is behind it.

One thing that changed in MLflow 3: the old model “stages” (Staging, Production) are deprecated, and the modern way to promote a model is a named alias. So instead of transition_model_version_stage(stage="Production"), you call set_registered_model_alias(alias="production", ...) and load with the models:/name@alias syntax. Here a teammate named Niranjan registers the model from a real run, gives it the staging alias, loads it back, then promotes it to production.

📄 model_registry.py: versioning and promoting models with aliases

import mlflow
import mlflow.sklearn
from mlflow import MlflowClient
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("iris-classification")

# First, train and log a model so we have a real run to register
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
with mlflow.start_run(run_name="best-model") as run:
    model = RandomForestClassifier(n_estimators=100, random_state=42)
    model.fit(X_train, y_train)
    mlflow.sklearn.log_model(model, name="model")
    run_id = run.info.run_id

client = MlflowClient()

# Niranjan registers the best model from that run
model_uri = f"runs:/{run_id}/model"
result = mlflow.register_model(model_uri, "iris-classifier")
print(f"Registered model: {result.name}, version: {result.version}")

# Promote with an alias (MLflow 3 way: aliases replace the old Staging/Production stages)
client.set_registered_model_alias(
    name="iris-classifier",
    alias="staging",
    version=result.version,
)
print(f"Model v{result.version} given alias 'staging'")

# Load model by alias for inference
staging_model = mlflow.sklearn.load_model("models:/iris-classifier@staging")
print(f"Loaded staging model: {type(staging_model).__name__}")

# After testing, promote to production
client.set_registered_model_alias(
    name="iris-classifier",
    alias="production",
    version=result.version,
)
print(f"Model v{result.version} given alias 'production'")

# List all versions with their aliases
registered = client.get_registered_model("iris-classifier")
alias_map = registered.aliases  # {alias_name: version}
for mv in client.search_model_versions("name='iris-classifier'"):
    aliases = [a for a, v in alias_map.items() if str(v) == str(mv.version)]
    print(f"  v{mv.version}: aliases={sorted(aliases)}, status={mv.status}")

▶ Output

Registered model: iris-classifier, version: 1
Model v1 given alias 'staging'
Loaded staging model: RandomForestClassifier
Model v1 given alias 'production'
  v1: aliases=['production', 'staging'], status=READY

What happened here: register_model took the model from a real run and gave it a name and a version number (version 1). Each call to set_registered_model_alias stuck a label on that version, first staging, then production. Loading with models:/iris-classifier@staging fetched the version behind the staging label and confirmed it is a RandomForestClassifier. Train a better model next week, register it as version 2, move the production alias to it, and your serving code that loads models:/iris-classifier@production picks up the new model with zero changes. To roll back, point the alias at the older version again. That is the real strength of the registry: humans decide which version is “the one”, machines just follow the label.

Version Your Data, Not Just Your Model

The whole point of a model version is that you can rebuild it. But a model is only half of the recipe. The other half is the exact data it trained on. Say you registered version 2 last month, and since then someone quietly fixed three rows in the training table. Retraining from “the same code” now gives you a different model, so version 2 is impossible to reproduce. Rollback is just as shaky: pointing the production alias back at an old model version does not save you if you cannot tell which dataset that version actually saw. A model version without its data version is a phone number with no name attached.

In practice you treat the dataset like any other input you log. Compute a content hash of the data (a checksum over the file, or over the rows), and store that hash alongside the run, right next to the parameters and metrics. In MLflow you can drop it in with mlflow.log_param("data_hash", ...) or mlflow.log_input(). The hash is tiny and cheap, but it works like a fingerprint: if two runs share the same data hash, they trained on byte-for-byte the same data, and if the hash changed, you know the data moved even when nobody told you. Now any run traces back to the precise rows that produced it, which is what “log the data version” really asks for.

For anything bigger than a toy CSV, the common tool is DVC (Data Version Control), and the idea behind it is neat. The large data files live in object storage (S3, Azure Blob, or a shared drive), and DVC keeps a small pointer file in git that records the hash and where the real data sits. Git stays fast because it only tracks the pointer, not the gigabytes, yet a git checkout of an old commit brings back the matching pointer, and dvc pull then fetches the exact data that went with it. Pair that with your MLflow run and you have the full chain: this alias points at this model version, which came from this run, which trained on this data hash. That chain is the difference between a model you can defend in an audit and one you just hope still works.

Common Mistakes

Practice Exercises

1. Exercise 1: Take the experiment tracking script and add a fourth model, a LogisticRegression. Log its parameters, metrics, and the model itself, then open mlflow ui and find which run scored best.
2. Exercise 2: Register the best run from Exercise 1 as version 2 of iris-classifier, then move the production alias onto it. Print the alias map before and after to confirm the swap.
3. Exercise 3: Write a tiny “rollback” function that takes a model name and a version number, points the production alias back at that version, and prints which version is now in production.

More in this series:

• Your First Kaggle Competition: An ML Project That Counts
• Debugging ML Models in Python: 5 Real Failure Cases
• ML: Best Practices, the Ethics and Catches Nobody Warns You About

Frequently Asked Questions

What’s Next?

You now know the two MLflow pieces you reach for every day: experiment tracking that records the settings, scores, and a saved copy of every run, and a model registry that versions those models and promotes them with aliases like staging and production. Put together, that means you can always answer “where did this model come from?” and roll back in seconds when a new model underperforms. On the Python MLOps ladder, that already puts you ahead of most teams. In the LLM evaluation tutorial we tackle the hardest MLOps problem for generative AI (GenAI): when the output is free-form text, how do you even measure whether an LLM did a good job?

Want the full picture? Browse the complete Python + AI/ML tutorial series home to see how experiment tracking fits alongside the data, training, and deployment tutorials.

Interview Questions on Python MLOps with MLflow

These come from real screens and onsites. Practice answering before you read each answer.

Q: What is the difference between MLflow Tracking and the Model Registry?

Tracking is the lab notebook: it logs the parameters, metrics, and a saved copy of the model for every training run, so you can compare experiments. The Registry sits on top of that: it takes a chosen run, gives it a name and a version number, and lets you promote versions with aliases like staging and production. In short, Tracking records everything you tried, and the Registry is where you pick and label the winners.

Q: Why do MLflow 3 aliases replace the old Staging and Production stages?

The old stages were a fixed, hard-coded set, so you were stuck with Staging, Production, Archived and nothing else. Aliases are free-form labels you define yourself, so you can add canary, champion, or challenger as needed. You promote with set_registered_model_alias() and load with models:/name@alias. Stages are deprecated in MLflow 3 and print a warning, so new code should use aliases.

Q: Why should serving code load a model by alias instead of by version number?

If your app hard-codes models:/iris-classifier/3, then every model swap is a code change and a redeploy. If it loads models:/iris-classifier@production instead, you promote a new version by moving the alias in the Registry and the running app picks it up with no code change. It also makes rollback trivial: point the production alias back at the older version.

Q: What does the tracking URI control, and why use SQLite locally?

The tracking URI tells MLflow where to store run metadata: parameters, metrics, and pointers to artifacts. sqlite:///mlflow.db writes to a single local file, which is perfect for learning and solo work and is what the Registry needs (a bare file store does not support model registration). For a team you would point it at PostgreSQL or MySQL with cloud artifact storage like S3 or Azure Blob so everyone shares one history.

Q: Your teammate promotes a new model to the production alias, but the live API keeps serving the old predictions. What do you check first?

Most often the app cached the model object at startup and never reloaded it, so moving the alias had no effect until a restart. Confirm the alias actually points at the new version with client.get_registered_model(), verify the serving process loads models:/name@production fresh (not a pickled copy baked into the image), and check that both point at the same tracking URI. A wrong or stale tracking URI is a classic cause of “it works on my machine”.

Q: Two runs use identical parameters but log different accuracy scores. How do you make the experiment reproducible?

The usual culprit is unfixed randomness or an unlogged input. Set and log a random_state for the train/test split and the model, then log the data version and the library versions, not just the score. MLflow captures parameters and metrics, but if the data or the seed changed underneath you, the numbers will drift. Logging everything that feeds the run is what turns “it scored 94% once” into a result you can rebuild on demand.

Further reading: MLflow documentation is the authoritative source on this.

Related Posts

Previous: Python ML Model Serving: From Notebook to Production Application Programming Interface (API)

Next: Model Drift in Python: Detect and Fix Decaying ML Models

Series Home: Python + AI/ML Tutorial Series