This pandas project takes a genuinely messy dataset and turns it into numbers you can trust. You will audit the mess first, fix it in clear stages (types, dates, duplicates, categories, outliers), validate the result, then answer three real business questions with groupby and merge. By the end you have a notebook you can put on GitHub and point a hiring manager at.

“Most of data science is janitor work. The model is the easy part, the cleaning is the actual job.”

Every working data analyst

Last Updated: July 2026 | Tested on: Python 3.14.6, Pandas 2.3.3 | Difficulty: Intermediate | Reading Time: 21 minutes

Tutorials love clean data. You call pd.read_csv, everything is already the right type, and the answer falls out in two lines. Real data is nothing like that. A real export has dates written three different ways, the word “Vegetables” spelled six different ways, duplicate rows from a botched re-sync, and empty cells hiding behind text like N/A and - so they do not even register as missing. If you run your analysis on that without looking first, you get an answer that is confidently wrong.

Think of it like cooking with groceries someone dumped on your counter unsorted. Some bags are mislabeled, a couple of items are spoiled, and one price tag says a single cake costs two lakh rupees. A careful cook checks everything before it goes in the pan. That check-first habit is the whole skill this post drills, and it is exactly what separates someone who “knows pandas” from someone a team actually trusts with data.

What We Are Building

We are cleaning a grocery store’s order export: 30 rows of orders with a date, city, category, item, quantity, and amount. It is small enough to read on screen but dirty in every way real data is dirty. The plan follows one repeatable shape you can reuse on any pandas project: audit first, fix in stages, validate, then analyze. The diagram below is that whole pipeline in one picture.

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

Notice the order. The tempting move is to jump straight to the fixing, but the audit step comes first for a reason: you cannot fix defects you have not named yet. We spend the first block just looking, writing down what is wrong, and only then start changing things. That single discipline is what keeps you from “fixing” a problem that was not actually there while missing the one that was.

Prerequisites

You should be comfortable with a DataFrame, selecting columns, and basic filtering from the Pandas introduction, and it helps to have seen merge and groupby since we use both at the end. Install the one library this pandas project needs with pip install pandas. Everything here was tested on Python 3.14.6 with Pandas 2.3.3. Every output block below is real output from running the code, not hand-typed examples, so if your numbers match, you did it right.

Step 1: Mirror the Messy Dataset

In a real job you would download this from a source like Kaggle and load it with pd.read_csv("orders.csv"). Here we generate the exact same mess in code and write it to a CSV. Doing it this way means the post always runs, and it can never break because some download link went dead years from now. Run this once to create messy_orders.csv in your folder.

📄 make_data.py: write a deliberately messy orders export

import pandas as pd

# A deliberately messy export from a grocery store's order system.
rows = [
    ["ORD-1001", "2026-03-01",  "Pune",   "Vegetables", "Spinach",     "3",   "120.50"],
    ["ORD-1002", "01/03/2026",  "mumbai ","vegetables", "Tomato",      "5",   "90.00"],
    ["ORD-1003", "March 2 2026","Nagpur",  "VEG",       "Potato",      "10",  "150.00"],
    ["ORD-1004", "2026-03-02",  " Pune",   "Fruits",    "Banana",      "6",   "72.00"],
    ["ORD-1005", "02/03/2026",  "MUMBAI",  "fruit",     "Apple",       "4",   "240.00"],
    ["ORD-1006", "2026-03-03",  "nagpur",  "Dairy",     "Paneer",      "2",   "N/A"],
    ["ORD-1007", "March 3 2026","Pune",    "dairy",     "Curd",        "3",   "60.00"],
    ["ORD-1008", "2026-03-03",  "Mumbai",  "Bakery",    "Bread",       "2",   "-"],
    ["ORD-1009", "03/03/2026",  "Nagpur ", "bakery",    "Cookies",     "1",   "45.00"],
    ["ORD-1010", "2026-03-04",  "pune",    "Veggies",   "Carrot",      "8",   "88.00"],
    ["ORD-1011", "2026-03-04",  "Mumbai",  "Vegetables","Onion",       "12",  "144.00"],
    ["ORD-1012", "04/03/2026",  "Nagpur",  "Fruits",    "Mango",       "5",   "500.00"],
    ["ORD-1013", "March 5 2026","Pune",    "fruit",     "Grapes",      "3",   "210.00"],
    ["ORD-1014", "2026-03-05",  "mumbai",  "Dairy",     "Milk",        "6",   "180.00"],
    ["ORD-1015", "2026-03-05",  "Nagpur",  "VEG",       "Cabbage",     "2",   "50.00"],
    ["ORD-1016", "05/03/2026",  "Pune ",   "Bakery",    "Muffin",      "4",   "160.00"],
    ["ORD-1017", "2026-03-06",  "Mumbai",  "vegetables","Peas",        "3",   "N/A"],
    ["ORD-1018", "March 6 2026","nagpur",  "Fruits",    "Orange",      "7",   "133.00"],
    ["ORD-1019", "2026-03-06",  "Pune",    "dairy",     "Butter",      "1",   "55.00"],
    ["ORD-1020", "06/03/2026",  "MUMBAI",  "Veggies",   "Beans",       "-2",  "40.00"],
    ["ORD-1021", "2026-03-07",  "Nagpur",  "bakery",    "Cake",        "1",   "250000.00"],
    ["ORD-1022", "2026-03-07",  "Pune",    "Fruits",    "Papaya",      "2",   "70.00"],
    ["ORD-1023", "07/03/2026",  "mumbai ", "VEG",       "Cauliflower", "999", "95.00"],
    ["ORD-1024", "2026-03-08",  "Nagpur",  "Dairy",     "Cheese",      "2",   "-50.00"],
    ["ORD-1025", "2026-03-08",  "Pune",    "fruit",     "Guava",       "4",   "84.00"],
    ["ORD-1026", "08/03/2026",  "Mumbai",  "vegetables","Brinjal",     "5",   ""],
    ["ORD-1027", "2026-03-09",  "nagpur ", "Bakery",    "Bun",         "6",   "48.00"],
    ["ORD-1028", "2026-03-09",  "Pune",    "Dairy",     "Ghee",        "1",   "620.00"],
]

df = pd.DataFrame(rows, columns=[
    "order_id", "order_date", "city", "category", "item", "quantity", "amount"
])

# Real exports have accidental duplicate rows from re-syncs. Add two.
df = pd.concat([df, df.iloc[[1, 9]]], ignore_index=True)
df.to_csv("messy_orders.csv", index=False)
print(f"Wrote messy_orders.csv with {len(df)} rows and {df.shape[1]} columns")

Run that once and you have the exact same 30-row file used for every number in this post. Every kind of real-world mess is in there on purpose: three date formats, city names in every casing with stray spaces, the category “Vegetables” written as VEG, Veggies, and vegetables, missing amounts hidden as N/A and -, a negative quantity, and a cake priced at two lakh. That last one is the kind of outlier that quietly wrecks a revenue report if you never look.

Step 2: Audit Before You Touch Anything

Before changing a single value, we interrogate the data. Three methods do almost all of the work here: .info() tells you the column types and how many values are missing, .describe() summarizes the numbers, and .value_counts() exposes messy text columns by listing every distinct value. Read this like a doctor reading a chart: you are building a list of what is wrong, not treating anything yet.

📄 audit.py: look first, change nothing

import pandas as pd

df = pd.read_csv("messy_orders.csv")

df.info()
print("\nCATEGORY value_counts (raw):")
print(df["category"].value_counts())
print("\nCITY value_counts (raw):")
print(df["city"].value_counts())

▶ Output

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 30 entries, 0 to 29
Data columns (total 7 columns):
 #   Column      Non-Null Count  Dtype
---  ------      --------------  -----
 0   order_id    30 non-null     object
 1   order_date  30 non-null     object
 2   city        30 non-null     object
 3   category    30 non-null     object
 4   item        30 non-null     object
 5   quantity    30 non-null     int64
 6   amount      27 non-null     object
dtypes: int64(1), object(6)
memory usage: 1.8+ KB

CATEGORY value_counts (raw):
category
vegetables    4
Fruits        4
Dairy         4
Bakery        3
VEG           3
fruit         3
Veggies       3
Vegetables    2
dairy         2
bakery        2
Name: count, dtype: int64

CITY value_counts (raw):
city
Pune       7
Nagpur     5
Mumbai     4
mumbai     3
nagpur     2
MUMBAI     2
pune       2
 Pune      1
Nagpur     1
mumbai     1
Pune       1
nagpur     1
Name: count, dtype: int64

What happened here: The audit already handed us a defect list. Look at amount: its dtype is object (text), not a number, and it shows only 27 non-null out of 30, so three values are already missing. The object type is the tell that something non-numeric is stuck in that column. Next, category has ten distinct spellings for what should be four or five real categories. And city is worse: “Nagpur” and “Pune” each appear on multiple lines because trailing spaces and different casing make pandas treat "Pune", " Pune", and "pune" as three separate cities. None of this is visible until you count. That is the whole point of auditing.

Here is our written defect list, straight from the audit:

• amount is text, not a number, and hides missing values as N/A, -, and blanks.
• order_date is text in three different formats.
• category has ten spellings for a handful of real categories.
• city has casing and whitespace variants of three real cities.
• There are duplicate rows, plus a negative quantity and a wildly large amount to investigate.

Step 3: Fix the Data in Stages

Now we fix the list, one defect at a time. The code below reads as one script that starts from the raw CSV and builds up the cleaned frame stage by stage. Doing it in small, named stages (rather than one giant unreadable chain) means you can print and check after each one, which is exactly how you catch a fix that did not do what you thought.

Fix the fake-missing values and number types

The read_csv call already converted N/A and empty cells to real NaN, because those are in pandas’ default missing-value list. But - is not on that list, so it stayed as text and poisoned the whole column into object type. The clean fix is pd.to_numeric with errors="coerce", which turns any value it cannot read into NaN and leaves the real numbers as floats.

📄 clean.py (stage 1): coerce amount to a real number

import pandas as pd

df = pd.read_csv("messy_orders.csv")

# read_csv already turned "N/A" and "" into NaN, but NOT "-".
# to_numeric with errors="coerce" turns anything non-numeric into NaN.
df["amount"] = pd.to_numeric(df["amount"], errors="coerce")

print("Missing values per column after coercion:")
print(df.isna().sum())
print("\namount dtype is now:", df["amount"].dtype)

▶ Output

Missing values per column after coercion:
order_id      0
order_date    0
city          0
category      0
item          0
quantity      0
amount        4
dtype: int64

amount dtype is now: float64

What happened here: The amount column is now a real float64, and the missing count went from 3 to 4 because the sneaky - finally registered as missing. This is why you never trust a column’s type until you have checked it: the difference between “3 missing” and “4 missing” is one fake marker that read_csv did not know about.

Parse the mixed date formats

The dates come in three shapes: 2026-03-01, 01/03/2026, and March 2 2026. Rather than write three parsers, pd.to_datetime with format="mixed" figures out each row on its own. We add dayfirst=True so an ambiguous 01/03/2026 is read as 1 March, matching the Indian day-month order rather than the American month-day one.

📄 clean.py (stage 2): one call for three date formats

df["order_date"] = pd.to_datetime(df["order_date"], format="mixed", dayfirst=True)
print("order_date dtype is now:", df["order_date"].dtype)
print("Date range:", df["order_date"].min().date(), "to", df["order_date"].max().date())

▶ Output

order_date dtype is now: datetime64[ns]
Date range: 2026-03-01 to 2026-03-09

What happened here: All three formats collapsed into one proper datetime64 column. Now you can sort by date, filter a week, or group by month, none of which work while dates are text. A word of honesty: format="mixed" is convenient but slower on large data, and it can guess wrong on truly ambiguous dates, so on a big file it is worth confirming the min and max dates make sense, exactly as we did here.

Drop the duplicate rows, with proof

Duplicate rows inflate every total you compute. We do not delete them blindly, though: we first count them so the change is visible and defensible.

📄 clean.py (stage 3): count, then drop

dupe_count = df.duplicated().sum()
print("Exact duplicate rows found:", dupe_count)
before = len(df)
df = df.drop_duplicates().reset_index(drop=True)
print(f"Rows: {before} -> {len(df)} after dropping {before - len(df)}")

▶ Output

Exact duplicate rows found: 2
Rows: 30 -> 28 after dropping 2

What happened here: Two full duplicate rows were removed, dropping the count from 30 to 28. Printing the before and after is not decoration, it is your evidence: if a reviewer asks why the row count changed, you can point at this line. If you only wanted to dedupe on the order id rather than whole identical rows, you would pass subset="order_id" instead.

Normalize the categories and cities

For cities, stripping spaces and title-casing is enough to collapse the variants. Categories need real judgement, because VEG and Veggies are not fixable by casing alone. So we build an explicit mapping dictionary that says exactly which messy label maps to which clean one. A mapping dict is self-documenting: anyone reading it sees your decisions in one place.

📄 clean.py (stage 4): a mapping dict makes decisions explicit

df["city"] = df["city"].str.strip().str.title()

category_map = {
    "vegetables": "Vegetables", "veg": "Vegetables", "veggies": "Vegetables",
    "fruits": "Fruits", "fruit": "Fruits",
    "dairy": "Dairy",
    "bakery": "Bakery",
}
df["category"] = df["category"].str.strip().str.lower().map(category_map)
print("Cities after cleanup:", sorted(df["city"].unique()))
print("Categories after mapping:", sorted(df["category"].unique()))
print("Any category left unmapped (NaN)?", df["category"].isna().any())

▶ Output

Cities after cleanup: ['Mumbai', 'Nagpur', 'Pune']
Categories after mapping: ['Bakery', 'Dairy', 'Fruits', 'Vegetables']
Any category left unmapped (NaN)? False

What happened here: Twelve city variants became three, and ten category spellings became four. The last check matters more than it looks: after a .map(), any value not in your dictionary becomes NaN, so a spelling you forgot would vanish silently. Printing isna().any() right after the map confirms every label found a home. If it printed True, you would know your dictionary is missing a case.

Decide on outliers, out loud

This is the stage people get wrong. The instinct is to silently delete anything weird. The professional move is to separate impossible values from merely extreme ones, and to write down what you did with each. A negative quantity or a negative amount is impossible, so it is a data error. A cake for 250000 or a single order of 999 cauliflowers is not impossible, just deeply suspicious, so we flag it for review instead of pretending it was never there.

📄 clean.py (stage 5): impossible vs extreme, both documented

print("quantity summary:\n", df["quantity"].describe()[["min", "50%", "max"]])
print("\namount summary:\n", df["amount"].describe()[["min", "50%", "max"]])

# Impossible: a negative quantity or amount cannot be real.
impossible = df[(df["quantity"] < 0) | (df["amount"] < 0)]
print("\nImpossible rows (negative qty or amount):")
print(impossible[["order_id", "item", "quantity", "amount"]].to_string(index=False))

# Extreme but not impossible: flag for review, do not delete blindly.
extreme = df[(df["quantity"] > 100) | (df["amount"] > 10000)]
print("\nExtreme rows flagged for review:")
print(extreme[["order_id", "item", "quantity", "amount"]].to_string(index=False))

# Decision, written down: negatives become NaN, extremes get a flag column.
df.loc[df["quantity"] < 0, "quantity"] = pd.NA
df.loc[df["amount"] < 0, "amount"] = pd.NA
df["is_suspect"] = (df["quantity"] > 100) | (df["amount"] > 10000)

▶ Output

quantity summary:
 min     -2.0
50%      3.5
max    999.0
Name: quantity, dtype: float64

amount summary:
 min       -50.0
50%        92.5
max    250000.0
Name: amount, dtype: float64

Impossible rows (negative qty or amount):
order_id   item  quantity  amount
ORD-1020  Beans        -2    40.0
ORD-1024 Cheese         2   -50.0

Extreme rows flagged for review:
order_id        item  quantity   amount
ORD-1021        Cake         1 250000.0
ORD-1023 Cauliflower       999     95.0

What happened here: The describe() line is the alarm bell: a minimum quantity of -2 and a maximum amount of 250000 both scream “look at me”. We isolated the two impossible rows and set those bad cells to NaN rather than guessing a replacement. The two extreme-but-possible rows got an is_suspect flag, which keeps them in the data but lets us exclude them from money math on purpose, with a paper trail. That is the difference between cleaning and cheating: every removed or altered value has a reason attached to it.

Step 4: Validate What You Cleaned

No pandas project is done until you prove the cleaning worked. We build the final analysis frame by dropping the suspect rows and any row still missing a quantity or amount, then re-run the same missing-value check from the audit. The numbers should now be boring, and boring is the goal.

📄 clean.py (stage 6): re-audit the cleaned frame

clean = df[~df["is_suspect"]].dropna(subset=["amount", "quantity"]).copy()
clean["quantity"] = clean["quantity"].astype(int)
print("Clean rows kept for analysis:", len(clean))
print("Remaining missing values:\n",
      clean[["order_date", "city", "category", "quantity", "amount"]].isna().sum())

▶ Output

Clean rows kept for analysis: 20
Remaining missing values:
 order_date    0
city          0
category      0
quantity      0
amount        0
dtype: int64

What happened here: We started with 30 raw rows and kept 20 for analysis. That is a big drop, and it should make you pause, which is the point. Two rows were duplicates, two were suspect extremes, and the rest were removed for missing amounts or quantities. Every column now reports zero missing values, so the frame is finally safe to compute on. We also used .copy() so pandas does not warn about writing to a slice, and cast quantity back to a clean integer now that the impossible value is gone.

Step 5: Answer Three Real Questions

Clean data exists to answer questions, so let us answer three. The first two are plain groupby aggregations. The third brings in a second table, a per-category profit margin, and uses merge to combine it, which is how you enrich your data with information that lives somewhere else.

📄 clean.py (stage 7): groupby for two questions, merge for the third

# Q1: total revenue by category
q1 = clean.groupby("category")["amount"].sum().sort_values(ascending=False)
print("Q1 revenue by category:\n", q1)

# Q2: average order amount by city
q2 = clean.groupby("city")["amount"].mean().round(2).sort_values(ascending=False)
print("\nQ2 average order value by city:\n", q2)

# Q3: merge a margin lookup table, then estimate profit by category
margins = pd.DataFrame({
    "category": ["Vegetables", "Fruits", "Dairy", "Bakery"],
    "margin_pct": [0.20, 0.25, 0.35, 0.40],
})
by_cat = clean.groupby("category", as_index=False)["amount"].sum()
profit = by_cat.merge(margins, on="category", how="left")
profit["est_profit"] = (profit["amount"] * profit["margin_pct"]).round(2)
profit = profit.sort_values("est_profit", ascending=False).reset_index(drop=True)
print("\nQ3 estimated profit by category:\n", profit.to_string(index=False))

▶ Output

Q1 revenue by category:
 category
Fruits        1309.0
Dairy          915.0
Vegetables     642.5
Bakery         253.0
Name: amount, dtype: float64

Q2 average order value by city:
 city
Mumbai    163.50
Nagpur    154.33
Pune      153.95
Name: amount, dtype: float64

Q3 estimated profit by category:
   category  amount  margin_pct  est_profit
    Fruits  1309.0        0.25      327.25
     Dairy   915.0        0.35      320.25
Vegetables   642.5        0.20      128.50
    Bakery   253.0        0.40      101.20

What happened here: The groupby lines split the data by a column, summed or averaged the amount inside each group, and sorted the result. The merge attached each category’s margin, after which the profit column is a simple multiply. Notice the story in Q3: Fruits and Dairy earn almost the same profit even though Dairy sells far less, because Dairy carries a fatter margin. You could never see that from revenue alone, which is exactly why we brought the second table in.

Finally, the deliverable a stakeholder actually reads is not a table, it is a short written summary. Here are the five findings from this cleaned data:

• Fruits lead revenue at 1309, more than five times Bakery’s 253.
• Profit tells a different story: once margins are applied, Fruits (327) and Dairy (320) are nearly tied, because Dairy’s 35% margin beats Fruits’ 25%.
• City barely matters for average order value: Mumbai 163.50, Nagpur 154.33, and Pune 153.95 are within a few rupees of each other.
• One in three raw rows was unusable: we kept 20 of 30 after removing duplicates, impossible values, and two data-entry errors.
• The audit paid for itself: the single fake 250000 cake order would have inflated total revenue by roughly 80 times if we had trusted the raw file.

Ship It: Acceptance Criteria and Publishing

A pandas project is only finished when it meets a bar you set in advance. Check every box before you call this done:

• Every column has the right dtype: dates are datetime64, amount is float64, quantity is int.
• No exact duplicate rows remain, and the drop is proven with a before-and-after count.
• Fake-missing markers (N/A, -, blanks) are real NaN, not text.
• Categories and cities collapse to a small, documented set.
• Every outlier decision is written in code, never silent.
• The notebook answers at least three questions and ends with a short findings list.
• It runs top to bottom with no errors on a fresh kernel.

Once it passes, publish it. Put the notebook and the CSV in a GitHub repository with a short README that states the question, the cleaning decisions, and the findings, so a hiring manager can understand it in 60 seconds. If you would rather not manage files, a Kaggle notebook hosts the code and dataset together and runs in the browser, which is a low-friction way to share a pandas project as a portfolio piece. Either way, the README is what gets read first, so spend real time on it.

One note on scale for later. Pandas is the right default here and for datasets up to a few million rows. When files grow into the tens of millions of rows and cleaning starts to feel slow, Polars (version 1.41.2 at the time of writing) does the same work with a faster engine and the same audit-fix-validate mindset carries straight over. The habit you built in this pandas project, not the specific library, is the part that lasts.

Common Mistakes

Mistake 1: Analyzing before auditing

If you jump straight to groupby, pandas will happily give you an answer built on ten spellings of “Vegetables” and a 250000 cake. It will not crash, which is what makes it dangerous. In any pandas project, run .info(), .describe(), and .value_counts() first and write down what is wrong. The audit is not overhead, it is the part that makes the answer trustworthy.

Mistake 2: Trusting the default missing-value detection

Beginners assume read_csv catches every kind of missing value. It catches many (N/A, empty cells) but not custom junk like -, none, or ?. Those survive as text and silently corrupt the column’s type. Either pass your own list with pd.read_csv(..., na_values=["-", "?", "none"]), or run pd.to_numeric(col, errors="coerce") to force the issue and reveal the true missing count.

Mistake 3: Deleting outliers silently

Quietly dropping every row that looks weird changes your totals with no record of why. Six months later nobody, including you, can explain the numbers. Separate impossible values (negatives where none can exist) from extreme-but-possible ones, handle each on purpose, and leave a comment or a flag column explaining the call. Document, do not disappear.

Frequently Asked Questions

Interview Questions on Pandas Data Cleaning

Interviewers rarely ask for definitions. They ask what happens in situations like these.

Q: Walk me through how you would start on a messy dataset you have never seen.

Audit before touching anything. I run df.info() for dtypes and non-null counts, df.describe() for numeric ranges, and value_counts() on text columns to expose inconsistent labels. From that I write an explicit defect list, then fix each item in a named stage and re-check after each one. Cleaning before auditing means you fix imaginary problems and miss the real ones.

Q: A numeric column loaded as object dtype. What happened and how do you fix it?

At least one value is text, usually a missing-value marker that read_csv did not recognize, such as a dash or the word none. I fix it with pd.to_numeric(col, errors="coerce"), which converts the valid numbers to float and turns the junk into NaN. That also reveals the true missing count, which is often higher than the raw file suggested.

Q: What is the difference between to_numeric(errors=”coerce”) and astype(float)?

astype(float) raises an error the moment it hits a value it cannot convert, so a single dash crashes the whole call. to_numeric with errors="coerce" is forgiving: it converts what it can and replaces the rest with NaN, which is exactly what you want on dirty data where you expect some bad values.

Q: How do you clean a categorical column with inconsistent spellings?

Normalize the obvious noise first with str.strip().str.lower(), then map the remaining variants to canonical labels with a dictionary and .map(). The dictionary documents every decision in one place. The key follow-up is checking isna().any() afterward, because any spelling missing from the dictionary silently becomes NaN.

Q: How should you handle outliers in a cleaning pipeline?

Separate impossible from extreme. A negative quantity or negative price is impossible, so it is a data error and I set it to NaN. An unusually large but physically possible value gets flagged for review, not deleted, because it might be real. Every action is documented in code so the change to the totals is explainable later.

Q: After cleaning, your row count dropped from 30 to 20. Is that a problem?

Not necessarily, but it must be explained. I would trace the drop: how many were exact duplicates, how many were removed as impossible or suspect, and how many were missing a required field. As long as each removal has a documented reason and the surviving rows are trustworthy, a smaller clean dataset beats a larger dirty one. What you never want is an unexplained drop.

Q: A teammate named Aditi merged a margins table onto the sales data and some profit values came out as NaN. What went wrong?

A category in the sales data has no matching row in the margins table, so a left join fills those margins with NaN and the profit multiply propagates it. Usually it is a spelling or casing mismatch between the two key columns. The fix is to normalize both category columns the same way before merging, and to add validate= or check for unmatched keys so the gap surfaces loudly instead of hiding in a NaN.

Reference: the complete, always-current details live in pandas official documentation.

Related Posts

Previous: Pandas Merge, Join, Concat: Combining DataFrames

Next: Python: Pandas vs Polars vs Dask, DataFrames Compared

Series Home: Python + AI/ML Tutorial Series