Feature engineering turns raw columns into predictive power. This guide shows you how to build interaction terms, polynomial features, datetime extractions, binning, log transforms, and domain-specific features that lift model performance, with every example run on real data.

“Coming up with features is difficult, time-consuming, requires expert knowledge. Applied machine learning is basically feature engineering.”

Andrew Ng, Stanford CS229

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0, Pandas 2.3.3 | Difficulty: Intermediate | Reading Time: 16 minutes

Think of your raw data like the ingredients in your fridge. A model cannot cook with whole vegetables sitting in the drawer. It needs them chopped, measured, and prepped. A column called “order_date” is a whole onion. Chop it into “day_of_week”, “is_weekend”, “month”, and “days_since_last_order”, and now the model can taste the seasonal patterns and the churn signals that were hidden inside that one date. That chopping and prepping is feature engineering: creating new columns that hand the algorithm knowledge it could never dig out on its own.

Here is the part beginners find surprising. The gap between an okay model and a great one is almost never the algorithm. It is the features. A plain logistic regression fed clever features will often beat a fancy random forest fed raw columns. That is why Kaggle winners talk about features far more than they talk about model architecture.

Say a data analyst named Vinay was predicting apartment prices with the obvious columns: area, bedrooms, bathrooms. His model stalled at an R-squared around 0.72. Then he added a few engineered features (price per square foot for the neighborhood, distance to the nearest metro station, the floor-to-total-floors ratio, and the age of the building) and the same model jumped to roughly 0.89. He did not change the algorithm at all. He just fed it better features. Your own numbers will depend on your data, but the lesson holds: better features beat fancier models almost every time.

Prerequisites

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

The diagram sorts feature engineering into three buckets. Creation builds new features from what you already have, like pulling day-of-week out of a date. Transformation runs a math function over a column, such as a log or a polynomial expansion. Encoding turns text categories into numbers the model can read. Notice the common thread: every bucket squeezes more signal out of the data you already collected, without going back to gather anything new. That is why feature engineering is so often the highest-leverage step in a project. A single well-built feature can lift accuracy more than swapping in a heavier algorithm.

Datetime Feature Extraction

A single datetime column is packed with hidden signals. Time of day drives website traffic. Day of week drives sales. Month drives demand for seasonal products. On its own a raw timestamp tells a model almost nothing, but pull it apart and that one column becomes five or six features the model can actually learn from. Think of it like a postal address: “411001” is just a number until you split it into city, area, and pin code, and suddenly it means something.

📄 datetime_features.py: extract multiple features from one date column

import pandas as pd
import numpy as np

rng = np.random.default_rng(42)

# Simulate order data
dates = pd.date_range("2025-01-01", periods=365, freq="D")
df = pd.DataFrame({
    "order_date": rng.choice(dates, 500),
    "amount": rng.uniform(100, 5000, 500).round(2)
})

# Extract datetime features
df["year"] = df["order_date"].dt.year
df["month"] = df["order_date"].dt.month
df["day_of_week"] = df["order_date"].dt.dayofweek  # 0=Mon, 6=Sun
df["is_weekend"] = df["day_of_week"].isin([5, 6]).astype(int)
df["quarter"] = df["order_date"].dt.quarter
df["day_of_month"] = df["order_date"].dt.day
df["is_month_start"] = df["order_date"].dt.is_month_start.astype(int)
df["is_month_end"] = df["order_date"].dt.is_month_end.astype(int)

print("Original: 1 datetime column → 8 numeric features")
print(f"\nSample:\n{df.head(3).to_string()}")
print(f"\nWeekend vs Weekday avg amount:")
print(f"  Weekend: ₹{df[df['is_weekend']==1]['amount'].mean():,.0f}")
print(f"  Weekday: ₹{df[df['is_weekend']==0]['amount'].mean():,.0f}")

▶ Output

Original: 1 datetime column → 8 numeric features

Sample:
  order_date   amount  year  month  day_of_week  is_weekend  quarter  day_of_month  is_month_start  is_month_end
0 2025-02-02  3294.46  2025      2            6           1        1             2               0             0
1 2025-10-10  4350.70  2025     10            4           0        4            10               0             0
2 2025-08-27  2324.09  2025      8            2           0        3            27               0             0

Weekend vs Weekday avg amount:
  Weekend: ₹2,518
  Weekday: ₹2,563

What happened here: One order_date column became eight features the model can learn from. The .dt accessor does the heavy lifting: it reads the year, month, quarter, and day, and answers yes-or-no questions like “is this a weekend?” or “is this the first of the month?”. Those boolean flags get cast to 1 and 0 with .astype(int) so the model sees plain numbers. In this random sample the weekend and weekday averages came out close (the data was generated with no real weekend effect), so do not read meaning into the gap.

On real sales data this is exactly the split that reveals whether your customers spend more on Saturdays. Note that dayofweek counts from 0 for Monday up to 6 for Sunday, which is why the weekend check looks for 5 and 6.

Mathematical Transforms

Some columns have lopsided distributions that throw linear models off. Income, price, and population are the usual suspects: a few huge values stretch a long tail to the right while most of the data bunches up on the left. A log transform squeezes that tail back in and pulls the shape closer to a normal bell curve, which is what linear models prefer. Picture a long line at a chai stall where most people order one cup but one office manager orders forty: the log scale brings that outlier back near the crowd so it stops dominating everything. Polynomial features solve a different problem. They let the model bend, so it can fit curved relationships a straight line would miss.

📄 math_transforms.py: log, polynomial, and interaction features

import numpy as np
import pandas as pd
from sklearn.preprocessing import PolynomialFeatures

rng = np.random.default_rng(42)

# Simulated house data
df = pd.DataFrame({
    "area_sqft": rng.integers(500, 5000, 100),
    "bedrooms": rng.integers(1, 6, 100),
    "distance_km": rng.uniform(0.5, 30, 100),
})

# Log transform: compresses skewed distributions
df["log_area"] = np.log1p(df["area_sqft"])  # log1p = log(1+x), handles zero
df["log_distance"] = np.log1p(df["distance_km"])

print("Log transform (handles right-skewed data):")
print(f"  Area: range {df['area_sqft'].min()}-{df['area_sqft'].max()}")
print(f"  Log area: range {df['log_area'].min():.2f}-{df['log_area'].max():.2f}")

# Interaction features: capture combined effects
df["area_per_bedroom"] = df["area_sqft"] / df["bedrooms"]
df["total_rooms_area"] = df["area_sqft"] * df["bedrooms"]

print(f"\nInteraction features:")
print(f"  Area per bedroom: {df['area_per_bedroom'].mean():.0f} sqft/bedroom")

# Polynomial features: capture non-linear relationships.
# Pass a DataFrame (not .values) so the real column names are kept.
poly = PolynomialFeatures(degree=2, include_bias=False, interaction_only=False)
X_small = df[["area_sqft", "bedrooms"]].head(3)
X_poly = poly.fit_transform(X_small)
print(f"\nPolynomial (degree=2) from 2 features → {X_poly.shape[1]} features:")
print(f"  Names: {poly.get_feature_names_out()}")
print(f"  Sample:\n{X_poly[0].astype(int)}")

▶ Output

Log transform (handles right-skewed data):
  Area: range 697-4890
  Log area: range 6.55-8.50

Interaction features:
  Area per bedroom: 1255 sqft/bedroom

Polynomial (degree=2) from 2 features → 5 features:
  Names: ['area_sqft' 'bedrooms' 'area_sqft^2' 'area_sqft bedrooms' 'bedrooms^2']
  Sample:
[   901      5 811801   4505     25]

What happened here: The log transform squeezed the area range from 697 to 4890 down to 6.55 to 8.50, a much gentler spread for a linear model to handle. The interaction feature area_per_bedroom captures something neither column says on its own: how roomy each bedroom is. A 4000 sqft flat with two bedrooms feels very different from a 4000 sqft flat with six, and that single ratio tells the model so.

The polynomial step turned 2 columns into 5 by adding the squared terms and the cross-product. Because we passed a DataFrame instead of a raw NumPy array, scikit-learn kept the real names (area_sqft^2, area_sqft bedrooms) instead of generic x0 x1 labels, which makes the output far easier to read. One warning though: polynomial features grow fast. With include_bias=False, 10 columns at degree 3 already balloon to 285 features. Reach for them only when you actually suspect a curved relationship.

Binning: Converting Numbers to Categories

Sometimes the exact number does not matter as much as the bucket it falls into. A bank does not really care whether you are 24 or 25, it cares whether you are “young”, “middle-aged”, or “senior” because each group behaves differently. Binning groups a continuous column into a handful of labelled ranges, the same way a shop sorts shirts into S, M, L, and XL instead of taping a measuring tape to every customer. Pandas gives you two tools for this. pd.cut slices on values you choose (everyone 0 to 25 is “Young”). pd.qcut slices so each bucket holds roughly the same number of people, which is handy when you want balanced groups.

📄 binning.py: create categorical features from continuous values

import pandas as pd
import numpy as np

rng = np.random.default_rng(42)
ages = rng.integers(18, 65, 20)
df = pd.DataFrame({"age": ages})

# Equal-width bins
df["age_group"] = pd.cut(df["age"], bins=[0, 25, 35, 50, 100],
                          labels=["Young", "Adult", "Middle", "Senior"])

# Quantile-based bins (equal number of samples per bin)
df["age_quantile"] = pd.qcut(df["age"], q=4, labels=["Q1", "Q2", "Q3", "Q4"])

print(df.sort_values("age").head(10).to_string())
print(f"\nAge group counts:\n{df['age_group'].value_counts().sort_index()}")

▶ Output

    age age_group age_quantile
0    22     Young           Q1
6    22     Young           Q1
9    22     Young           Q1
17   24     Young           Q1
8    27     Adult           Q1
4    38    Middle           Q2
3    38    Middle           Q2
19   39    Middle           Q2
10   42    Middle           Q2
16   42    Middle           Q2

Age group counts:
age_group
Young     4
Adult     1
Middle    7
Senior    8
Name: count, dtype: int64

What happened here: Each age landed in two new categorical columns. pd.cut read the explicit edges we gave it, so anyone 0 to 25 became “Young” and anyone above 50 became “Senior”. pd.qcut ignored fixed edges and instead chopped the ages into four equal-sized groups, Q1 through Q4. With only 20 random ages the age_group counts came out uneven (just one “Adult” but eight “Senior”), which is the catch with pd.cut: your chosen edges do not promise balanced buckets. If you need every bucket to hold a similar number of rows, pd.qcut is the safer choice. Your exact counts will differ from a different random seed.

Domain-Specific Feature Engineering

This is where feature engineering stops being generic tricks and starts being your edge. The best features come from knowing the business. In e-commerce, the classic trio is RFM: Recency (how long since the customer last bought), Frequency (how many times they bought), and Monetary (how much they spent in total). A shopkeeper who knows her regulars does this in her head.

Picture her thinking about a longtime customer named Pravin: “He used to come every week, now I have not seen him in a month, and he always spent big.” That sentence is exactly recency, frequency, and monetary, and it is also a churn warning. The code below builds all three from a raw transaction log with a single groupby.

📄 domain_features.py: real-world feature engineering patterns

import pandas as pd
import numpy as np

rng = np.random.default_rng(42)

# E-commerce: customer purchase history
purchases = pd.DataFrame({
    "customer_id": rng.choice(50, 200),
    "order_date": pd.date_range("2024-06-01", periods=200, freq="2D")[:200],
    "amount": rng.uniform(100, 5000, 200).round(2),
})

# RFM features (Recency, Frequency, Monetary): standard for churn prediction
reference_date = purchases["order_date"].max()
rfm = purchases.groupby("customer_id").agg(
    recency=("order_date", lambda x: (reference_date - x.max()).days),
    frequency=("order_date", "count"),
    monetary=("amount", "sum")
)

print("RFM Features (customer-level aggregation):")
print(rfm.sort_values("monetary", ascending=False).head(5))
print(f"\nGenerated 3 powerful features from raw transaction data.")
print(f"Recency, frequency, and monetary are the classic churn signals.")

▶ Output

RFM Features (customer-level aggregation):
             recency  frequency  monetary
customer_id
4                  8          8  26568.07
21                90          7  21402.22
33                76          6  19829.76
34                18          7  19533.73
39                80          9  18898.35

Generated 3 powerful features from raw transaction data.
Recency, frequency, and monetary are the classic churn signals.

What happened here: One groupby("customer_id") rolled hundreds of raw transactions up into one row per customer, with three features each. The recency column uses a small lambda to measure the days between each customer’s last order and the most recent date in the whole dataset, so a low number means “bought recently”. frequency just counts their orders, and monetary sums their spending. Customer 4 looks like a star: bought 8 times, very recently, and spent the most. Customer 21 spent well too but has not ordered in 90 days, which is the kind of recency gap that flags a churn risk worth a follow-up.

Three columns, built from a transaction log you already had, that a churn model can lean on hard. Your exact numbers will shift with a different random seed.

Common Mistakes

The most expensive feature engineering mistake has a name: target leakage. It happens when a feature secretly contains information you would not actually have at prediction time. The model looks brilliant in testing, then falls apart in production. Imagine studying for an exam with a copy of the answer key, scoring 100 percent, and then sitting the real exam without it. That gap between practice and reality is exactly what leakage does to a model.

❌ Mistake: creating features that leak the target

# BAD: "avg_price_in_category" computed from ALL data (including the test set).
# This is target leakage: the feature smuggles in information about
# the very thing you are trying to predict.

# BAD: using future data to predict the past.
# If you are predicting January sales, do not feed in February metrics.

# GOOD: only use information you would actually have at prediction time.
# "avg_price_in_category" computed from the TRAINING set only.
# Time-based features: only look backward, never forward.
print("Target leakage is the #1 cause of models that work in testing")
print("but fail in production. Always ask: 'Would I have this")
print("information at the time I need to make the prediction?'")

▶ Output

Target leakage is the #1 cause of models that work in testing
but fail in production. Always ask: 'Would I have this
information at the time I need to make the prediction?'

Why this bites: the fix is one habit. Compute every aggregated feature (group averages, counts, encodings) on the training data only, then apply those learned values to the validation and test sets. The same rule covers time: a feature built for a given day may only use data from that day or earlier, never later. If you can honestly answer “yes, I would have this number at the moment I make the prediction”, the feature is safe. If you are not sure, treat it as leakage until you prove otherwise.

Try It Yourself

Take the datetime example from earlier and build on it. The goal is to feel how much signal you can squeeze out of plain columns.

1. Exercise 1: From the order_date column, add a days_since_first_order feature (subtract the earliest date in the data from each row). This is one of the strongest churn signals you can build.
2. Exercise 2: Bin the amount column into “Low”, “Medium”, and “High” spenders with pd.qcut, then check how many orders land in each bucket.
3. Exercise 3: Build a single interaction feature that you think predicts a big order, for example is_weekend times amount, and explain in one sentence why it might help.

Conclusion

You now have a full toolkit for turning raw columns into predictive power: pulling year, month, and weekend flags out of a single datetime; taming skewed columns with log transforms; bending straight lines with polynomial and interaction features; grouping numbers into buckets with pd.cut and pd.qcut; and building domain features like RFM that carry real business insight. The thread running through all of it is simple: better features usually beat a fancier model, and every technique here squeezes more signal from data you already collected. The one habit that saves you the most pain is guarding against target leakage, so keep asking whether you would truly have a feature at prediction time.

Next comes the natural follow-on: once you have generated dozens of features, you need to keep only the ones that matter. That is feature selection, where you drop the noise and keep the signal. For the complete path from Python basics to deployed ML models, head to the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions on Feature Engineering

If you can walk through these without peeking, you are ready for this topic in an interview.

Q: What is target leakage, and how do you prevent it when engineering features?

Target leakage happens when a feature secretly carries information you would not have at prediction time, so the model scores brilliantly in testing and then fails in production. The classic cause is computing an aggregate (a group mean, a target encoding) over the full dataset instead of the training split. Prevent it by fitting every learned statistic on the training set only, then applying those values to validation and test, and by making sure time-based features only look backward.

Q: Your model gets 0.98 Area Under the Curve (AUC) in cross-validation but drops to 0.61 in production. What feature engineering issue do you check first?

A gap that large almost always points to target leakage rather than a modelling problem. Check whether any feature was built using information from the future or from the test rows, for example a category average or a target encoding computed on the entire dataset before splitting. Rebuild those features inside a pipeline that fits only on the training fold, and re-run the validation. If the inflated score disappears, you found the leak.

Q: You have a high-cardinality categorical column such as pincode with 5,000 distinct values. How would you engineer it?

One-hot encoding would explode into 5,000 sparse columns, so reach for target encoding or frequency encoding instead, which collapse the column into one or two numeric features. Fit the encoding on the training set only and apply the learned mapping to validation and test to avoid leakage, and add smoothing so rare pincodes do not overfit. You can also group values by a meaningful hierarchy, such as mapping pincode to city or region, which often carries most of the signal with far fewer categories.

Q: When would you use a log transform, and why is np.log1p often safer than np.log?

Use a log transform on right-skewed columns like income, price, or population, where a few huge values stretch a long tail and throw linear models off. The log compresses that tail and pulls the distribution closer to a bell curve. np.log1p computes log(1 + x), so it handles zeros gracefully where plain np.log(0) would return negative infinity, which makes it the safer default for counts and amounts that can be zero.

Q: What is the difference between pd.cut and pd.qcut, and when do you pick each?

pd.cut splits on edges you choose, so you control the exact boundaries (for example ages 0 to 25 become “Young”), but the buckets can end up very unbalanced. pd.qcut splits on quantiles so each bucket holds roughly the same number of rows, which is better when you want balanced groups for a model. Use pd.cut when the boundaries have real-world meaning and pd.qcut when even bucket sizes matter more than the exact cutoffs.

Q: Why can polynomial features hurt more than they help, and how do they scale?

Polynomial features add squared terms and cross-products, so the count grows combinatorially: with include_bias=False, 10 columns at degree 3 balloon to 285 features and 20 columns to 1,770. That inflates training time and invites overfitting, especially on small datasets. Use them only when you genuinely suspect a curved relationship, and skip them entirely for tree-based models like random forests, which capture non-linearity on their own.

Series: Python + AI/ML Cookbook, Part 5: Machine Learning

Go deeper: when you outgrow this post, the official Python documentation is the next stop.

Related Posts

Previous: ML: Data Preprocessing, Your Model Is Only as Good as Your Data

Next: ML: Feature Selection, Drop the Noise, Keep the Signal

Series Home: Python + AI/ML Tutorial Series