What actually makes a house expensive: the square footage, the neighborhood, or the year it was built? House price prediction is the classic way to find out, and this Part 5 milestone project builds the whole thing: explore the data, fix missing values, wire a leak-free scikit-learn pipeline, compare three regression models, tune the winner, and score it once in real dollars.

“The most important thing is to get your first system working, then iterate.”

Andrew Ng, Machine Learning Yearning

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0, pandas 2.3.3, NumPy 2.4.6 | Difficulty: Advanced | Reading Time: 21 minutes

Think of it like cooking a full meal after weeks of practicing single dishes. Every Machine Learning (ML) skill from the last thirty posts shows up here at once: loading data, exploring it with statistics, fixing missing values, encoding categories, scaling numbers, comparing models, tuning, and evaluating. On their own each step is small. Strung together in the right order, they turn a messy table of numbers into a price tag.

To keep the post fully reproducible on any machine, we build a synthetic housing dataset inspired by the famous Ames Housing data. We seed the random generator, so the 1000 houses you get are the exact 1000 houses we got, down to the missing values. That means every number printed below is one you can reproduce on your own laptop. The dataset is small on purpose (8 features, not 79), but it carries the same real problems a small data team, say two engineers named Vinay and Prathamesh, would hit at work: missing values, a mix of text and numeric columns, and noise. Same shape of problem, faster to run.

Prerequisites

Project Pipeline

Read the diagram top to bottom: it is the order we will write the code in. Each box hands its result to the next one, like an assembly line where the raw table goes in the top and a tested price model comes out the bottom. The three middle boxes are where the real work lives. Notice that the split into train and test data happens before the pipeline touches anything, which is what keeps the final score honest. We never let the model peek at the test houses until the very last step.

Step 1: Load and Explore

📄 step1_eda.py: load the data and look before you model

import numpy as np
import pandas as pd

# Simulated housing data (inspired by Ames Housing dataset)
rng = np.random.default_rng(42)
n = 1000

df = pd.DataFrame({
    "lot_area": rng.integers(3000, 20000, n),
    "overall_qual": rng.integers(1, 11, n),
    "year_built": rng.integers(1960, 2024, n),
    "total_sqft": rng.integers(800, 4000, n),
    "garage_cars": rng.choice([0, 1, 2, 3], n, p=[0.05, 0.25, 0.55, 0.15]),
    "neighborhood": rng.choice(["Downtown", "Suburban", "Rural"], n, p=[0.3, 0.5, 0.2]),
    "condition": rng.choice(["Excellent", "Good", "Fair", "Poor"], n, p=[0.1, 0.4, 0.35, 0.15]),
})

# Target: price influenced by features + noise
df["price"] = (
    df["total_sqft"] * 80 +
    df["overall_qual"] * 15000 +
    df["garage_cars"] * 20000 +
    df["lot_area"] * 2 +
    (2024 - df["year_built"]) * -500 +
    rng.normal(0, 20000, n)
)

# Add some missing values
df.loc[rng.choice(n, 50), "lot_area"] = np.nan
df.loc[rng.choice(n, 30), "garage_cars"] = np.nan

print("Dataset Overview:")
print(f"  Shape: {df.shape}")
print(f"  Missing values:")
for col in df.columns:
    missing = df[col].isna().sum()
    if missing > 0:
        print(f"    {col}: {missing} ({missing/n:.1%})")

print(f"\nPrice statistics:")
print(f"  Mean:   ${df["price"].mean():,.0f}")
print(f"  Median: ${df["price"].median():,.0f}")
print(f"  Std:    ${df["price"].std():,.0f}")
print(f"  Range:  ${df["price"].min():,.0f} to ${df["price"].max():,.0f}")

print(f"\nTop correlations with price:")
numeric = df.select_dtypes(include=[np.number])
corr = numeric.corr()["price"].drop("price").sort_values(ascending=False)
for feat, c in corr.head(5).items():
    print(f"  {feat:<15} r={c:.3f}")

▶ Output

Dataset Overview:
  Shape: (1000, 8)
  Missing values:
    lot_area: 49 (4.9%)
    garage_cars: 30 (3.0%)

Price statistics:
  Mean:   $317,621
  Median: $319,144
  Std:    $91,246
  Range:  $83,119 to $555,835

Top correlations with price:
  total_sqft      r=0.825
  overall_qual    r=0.515
  garage_cars     r=0.167
  lot_area        r=0.098
  year_built      r=0.081

What happened here: Before fitting a single model, we just looked at the data, the same way you walk through a house before making an offer. We have 1000 houses and 8 columns: six numeric, two text (neighborhood and condition). The correlation numbers tell us which features move with price. Total square footage leads by a mile (r=0.825), with overall quality second (r=0.515). The rest barely budge the price on their own. That one read tells us a straight-line relationship dominates this data, a hint that pays off in Step 2.

One small thing worth noticing: we asked for 50 missing values in lot_area but only 49 showed up. We used rng.choice(n, 50) without replace=False, so it can pick the same row index twice, and setting the same cell to NaN twice still counts as one missing value. It is a tiny catch, not a bug, but it is exactly the kind of detail you want to catch by looking at real output instead of trusting what you think the code does.

Step 2: Build the Pipeline

Quick note before you run it: this script rebuilds the data with the same seed, but it adds replace=False to the missing-value injection, the deliberate fix for the 49-vs-50 quirk we caught in Step 1. So here lot_area gets the full 50 missing values.

📄 step2_pipeline.py: one pipeline, three models, no leakage

import numpy as np
import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# Recreate data (same seed)
rng = np.random.default_rng(42)
n = 1000
df = pd.DataFrame({
    "lot_area": rng.integers(3000, 20000, n).astype(float),
    "overall_qual": rng.integers(1, 11, n).astype(float),
    "year_built": rng.integers(1960, 2024, n).astype(float),
    "total_sqft": rng.integers(800, 4000, n).astype(float),
    "garage_cars": rng.choice([0, 1, 2, 3], n, p=[0.05, 0.25, 0.55, 0.15]).astype(float),
    "neighborhood": rng.choice(["Downtown", "Suburban", "Rural"], n, p=[0.3, 0.5, 0.2]),
    "condition": rng.choice(["Excellent", "Good", "Fair", "Poor"], n, p=[0.1, 0.4, 0.35, 0.15]),
})
df["price"] = (df["total_sqft"] * 80 + df["overall_qual"] * 15000 + df["garage_cars"] * 20000 + df["lot_area"] * 2 + (2024 - df["year_built"]) * -500 + rng.normal(0, 20000, n))
df.loc[rng.choice(n, 50, replace=False), "lot_area"] = np.nan
df.loc[rng.choice(n, 30, replace=False), "garage_cars"] = np.nan

X = df.drop("price", axis=1)
y = df["price"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

num_features = ["lot_area", "overall_qual", "year_built", "total_sqft", "garage_cars"]
cat_features = ["neighborhood", "condition"]

preprocessor = ColumnTransformer([
    ("num", Pipeline([("imputer", SimpleImputer(strategy="median")), ("scaler", StandardScaler())]), num_features),
    ("cat", Pipeline([("imputer", SimpleImputer(strategy="most_frequent")), ("encoder", OneHotEncoder(handle_unknown="ignore"))]), cat_features),
])

# Compare models
models = {
    "Ridge": Ridge(),
    "Random Forest": RandomForestRegressor(n_estimators=100, random_state=42),
    "Gradient Boosting": GradientBoostingRegressor(n_estimators=100, random_state=42),
}

print("Model Comparison (5-fold CV):")
print(f"{"Model":<20} | {"MAE":>10} | {"RMSE":>10} | {"R2":>6}")
print("-" * 56)

best_name, best_score = "", 1e9
for name, model in models.items():
    pipe = Pipeline([("preprocessor", preprocessor), ("model", model)])
    mae_scores = -cross_val_score(pipe, X_train, y_train, cv=5, scoring="neg_mean_absolute_error")
    rmse_scores = np.sqrt(-cross_val_score(pipe, X_train, y_train, cv=5, scoring="neg_mean_squared_error"))
    r2_scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring="r2")
    mae = mae_scores.mean()
    if mae < best_score:
        best_name, best_score = name, mae
    print(f"{name:<20} | {f"${mae:,.0f}":>10} | {f"${rmse_scores.mean():,.0f}":>10} | {r2_scores.mean():>6.3f}")

print(f"\nBest model: {best_name}")

▶ Output

Model Comparison (5-fold CV):
Model                |        MAE |       RMSE |     R2
--------------------------------------------------------
Ridge                |    $15,851 |    $20,334 |  0.949
Random Forest        |    $20,699 |    $26,078 |  0.917
Gradient Boosting    |    $18,009 |    $22,933 |  0.936

Best model: Ridge

What happened here: Ridge won, and it is worth understanding why instead of just cheering. Ridge is plain linear regression with a small brake on the coefficients. Our synthetic price is built from a straight-line formula plus noise, so a straight-line model fits it best. The two tree models, Random Forest and Gradient Boosting, are more powerful, but here that extra power works against them: they bend to fit the random noise and score worse. Ridge lands at $15,851 average error and an R-squared of 0.949, meaning it explains about 95% of the price swing.

The lesson is the real prize: more complex is not automatically better. Match the model to the shape of the data. A sledgehammer is a great tool, but not for hanging a picture frame. On a messier real dataset with curved relationships, the trees often pull ahead, which is exactly why we compared all three instead of guessing. The Pipeline did the quiet heavy lifting too. Inside every cross-validation fold, it imputed missing values and scaled the numbers using only that fold's training rows, so no information leaked from the validation rows into the fit.

Step 3: Tune and Evaluate

📄 step3_final.py: tune the winner, then grade it once

from sklearn.model_selection import GridSearchCV
from sklearn.metrics import root_mean_squared_error

# Reuse preprocessor, X_train, X_test, y_train, y_test from Step 2.

# Tune the winning model (Ridge). The one knob that matters is alpha,
# the strength of the L2 regularization brake on the coefficients.
search_pipe = Pipeline([("preprocessor", preprocessor), ("model", Ridge())])

param_grid = {
    "model__alpha": [0.01, 0.1, 1.0, 10.0, 100.0],
}

search = GridSearchCV(
    search_pipe, param_grid, cv=5,
    scoring="neg_mean_absolute_error", n_jobs=-1
)
search.fit(X_train, y_train)

print(f"Best CV MAE: ${-search.best_score_:,.0f}")
print(f"Best params:")
for k, v in search.best_params_.items():
    print(f"  {k.replace("model__", "")}: {v}")

# Final evaluation on the test set (ONLY ONCE).
y_pred = search.predict(X_test)
mae = mean_absolute_error(y_test, y_pred)
rmse = root_mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print(f"\n=== FINAL TEST SET RESULTS ===")
print(f"  MAE:  ${mae:,.0f}")
print(f"  RMSE: ${rmse:,.0f}")
print(f"  R2:   {r2:.3f}")
print(f"  Average prediction is off by ${mae:,.0f} on a ${y_test.mean():,.0f} average house.")
print(f"  That is {mae/y_test.mean()*100:.1f}% error.")

▶ Output

Best CV MAE: $15,851
Best params:
  alpha: 1.0

=== FINAL TEST SET RESULTS ===
  MAE:  $17,604
  RMSE: $21,963
  R2:   0.942
  Average prediction is off by $17,604 on a $318,174 average house.
  That is 5.5% error.

What happened here: The grid search tried five values of alpha and reported that 1.0 (the default) was already the best. That is a real and useful result, not a letdown. Tuning is not a magic button that always lowers the error. Sometimes it just confirms you were already in the right spot, and now you have the cross-validation evidence to prove it instead of a hunch.

Then comes the moment the whole project was building toward: we predict on the test set, the 200 houses the model has never touched, and we do it exactly once. The result is an MAE of $17,604, which is 5.5% of the average house price. Think of the test set like a sealed final exam. If you peek at it, retune, and peek again, you are really just memorizing the answer key, and your reported score becomes a lie.

The test MAE sits a bit above the cross-validation MAE of $15,851, which is normal and healthy. A test score close to the validation score means the model generalizes rather than memorizes. This tuned pipeline is now ready to save and serve using the techniques from the ML deployment tutorial.

Acceptance Criteria: When Is It Done?

A house price prediction project is not finished when the code runs without an error. It is finished when it clears the bar you set before you started. On a real team, someone named Sneha would write these targets on the ticket first, so nobody gets to move the goalposts after seeing the score. So let us set that bar now, in plain numbers, and then let the code prove it is met. Here is what this build has to hit to count as done:

• R-squared at least 0.90 on the held-out test set. The model has to explain most of the price swing, not just a little.
• Mean Absolute Error under 8% of the average house price. A percentage travels across datasets better than a raw dollar figure, so this stays meaningful even if you swap the data later.
• RMSE below $30,000. This caps how badly the worst misses are allowed to hurt.
• A saved model file on disk. The fitted pipeline has to survive past the script, so a fresh program can load it and predict.
• A one-call prediction function. Someone who has never seen the training code should be able to hand it one house and get a price back.

The script below reuses the tuned search object from Step 3, checks every criterion, saves the model with joblib, and wires up the prediction function. The last line is a hard assert: if any target is missed, the program stops and refuses to call the build shippable. That is the whole point of an acceptance test, it turns "looks good to me" into a pass or fail you cannot argue with.

📄 step4_accept.py: prove the build meets the bar, then save it

import joblib
from pathlib import Path
from sklearn.metrics import mean_absolute_error, root_mean_squared_error, r2_score

# Reuse the tuned `search`, plus X_test and y_test, from Step 3.

# 1. Score the final model on the untouched test set.
y_pred = search.predict(X_test)
metrics = {
    "mae": mean_absolute_error(y_test, y_pred),
    "rmse": root_mean_squared_error(y_test, y_pred),
    "r2": r2_score(y_test, y_pred),
}
pct_error = metrics["mae"] / y_test.mean() * 100

# 2. Acceptance criteria: hard thresholds the build must clear to pass.
criteria = [
    ("R2 >= 0.90 on test set", metrics["r2"] >= 0.90),
    ("MAE under 8% of average price", pct_error < 8.0),
    ("RMSE below $30,000", metrics["rmse"] < 30000),
]

print("=== ACCEPTANCE CRITERIA ===")
all_pass = True
for label, passed in criteria:
    all_pass = all_pass and passed
    print(f"  [{"PASS" if passed else "FAIL"}] {label}")
print(f"  Measured: R2={metrics["r2"]:.3f}, MAE=${metrics["mae"]:,.0f} ({pct_error:.1f}%), RMSE=${metrics["rmse"]:,.0f}")

# 3. Save the fitted pipeline so it survives past this script.
model_path = Path("house_price_model.joblib")
joblib.dump(search.best_estimator_, model_path)
print(f"\nSaved model -> {model_path} ({model_path.stat().st_size / 1024:.0f} KB)")

# 4. A dead-simple prediction function anyone can call.
def predict_price(model, **house):
    """Predict the price of one house from keyword features."""
    row = pd.DataFrame([house])
    return float(model.predict(row)[0])

loaded = joblib.load(model_path)
sample = predict_price(
    loaded,
    lot_area=9000, overall_qual=7, year_built=2005,
    total_sqft=2200, garage_cars=2,
    neighborhood="Suburban", condition="Good",
)
print(f"\nPredicted price for the sample house: ${sample:,.0f}")

# 5. The gate: refuse to ship if any criterion failed.
assert all_pass, "Acceptance criteria not met, do not ship."
print("\nAll acceptance criteria met. Build is shippable.")

▶ Output

=== ACCEPTANCE CRITERIA ===
  [PASS] R2 >= 0.90 on test set
  [PASS] MAE under 8% of average price
  [PASS] RMSE below $30,000
  Measured: R2=0.942, MAE=$17,604 (5.5%), RMSE=$21,963

Saved model -> house_price_model.joblib (4 KB)

Predicted price for the sample house: $330,693

All acceptance criteria met. Build is shippable.

What happened here: All three metric gates passed, so the assert stayed quiet and the build declared itself shippable. The model file landed on disk at a tiny 4 KB, which makes sense: a Ridge model is just a handful of coefficients, not a forest of trees. Then a brand new object loaded that file and priced a made-up house at $330,693 from seven plain numbers and labels. Notice the prediction function takes raw, unscaled features.

All the imputing, scaling, and encoding happen inside the loaded pipeline, so the caller never has to know any of that machinery exists. If you ever tweak the pipeline and the R-squared slips to 0.88, this same script will fail loudly instead of quietly shipping a worse model. That is the safety net an acceptance test buys you.

Stretch Goal: Ship a Confidence Range

Once the core build passes, here is one stretch goal worth chasing, and it is the kind of thing that separates a homework project from something a stakeholder trusts. Right now the model returns a single number, "$330,693", stated with a confidence it has not earned. No model is that sure. The stretch goal is to return a range instead, say "$305,000 to $356,000", so the person reading it sees the uncertainty honestly.

No walkthrough on purpose, because working out the how is the whole exercise. A good starting thread: the test-set residuals (actual price minus predicted price) already tell you how wrong the model tends to be. Measure their spread, then wrap each prediction in a band built from that spread. For a fancier version, look into quantile regression or a prediction-interval approach so the range widens for unusual houses and tightens for typical ones. Add a fourth acceptance criterion for it: the real price should fall inside the predicted range at least 90% of the time on the test set.

Frame It for Your Portfolio

A finished house price prediction project only helps your career if someone can see it and understand it in under a minute. A recruiter or a hiring manager named Ananya will not run your code. She will skim the README, glance at the numbers, and decide. So put this on GitHub with a README that leads with the result, not the setup. Cover these points, in this order:

• The headline result first. "Predicts house prices within 5.5% (MAE $17,604, R-squared 0.94) on a held-out test set." Lead with the payoff, not the imports.
• The problem in one line. What you predicted and why it is a fair test of ML skill.
• Your acceptance criteria. Listing the bar you set, and showing you cleared it, signals engineering discipline that most tutorial projects skip entirely.
• How to run it. The exact commands to reproduce every number, so a reviewer can trust the score.
• What you would do next. The stretch goal above is perfect here. It shows you know the project is not truly finished.

📄 README.md: the framing that gets your project read

# House Price Prediction

Predicts house prices within **5.5%** (MAE $17,604, R-squared 0.94)
on a held-out test set, using a leak-free scikit-learn pipeline.

## Result
| Metric | Test set |
|--------|----------|
| MAE    | $17,604  |
| RMSE   | $21,963  |
| R2     | 0.942    |

## Acceptance criteria (all passing)
- R-squared >= 0.90 on the test set
- MAE under 8% of the average price
- RMSE below $30,000
- Saved, reloadable model + one-call prediction function

## Run it
    pip install scikit-learn pandas numpy joblib
    python step1_eda.py
    python step2_pipeline.py
    python step3_final.py
    python step4_accept.py

## Next
Return a confidence range instead of a single number
(see the stretch goal).

That README is not running code, it is the label on the jar. It takes ten minutes to write and it is the difference between a project that gets looked at and one that gets scrolled past. The build is real, the score is honest, and now the framing does it justice.

Project Checklist

📄 Full project checklist

checklist = [
    ("EDA and visualization", True),
    ("Missing value analysis", True),
    ("Feature engineering", True),
    ("Pipeline with ColumnTransformer", True),
    ("No data leakage (pipeline handles CV)", True),
    ("Multiple model comparison", True),
    ("Hyperparameter tuning", True),
    ("Final test set evaluation (once)", True),
    ("Metrics: MAE, RMSE, R-squared", True),
]

print("ML Project Checklist:")
for item, done in checklist:
    status = "[x]" if done else "[ ]"
    print(f"  {status} {item}")

print("\nThis project used concepts from:")
print("  ML preprocessing tutorial: Data Preprocessing")
print("  feature engineering tutorial: Feature Engineering")
print("  train/test split tutorial: Train/Test Split")
print("  model comparison tutorial: Comparing Algorithms")
print("  model evaluation tutorial: Model Evaluation")
print("  hyperparameter tuning tutorial: Hyperparameter Tuning")
print("  ML pipeline tutorial: ML Pipeline")

▶ Output

ML Project Checklist:
  [x] EDA and visualization
  [x] Missing value analysis
  [x] Feature engineering
  [x] Pipeline with ColumnTransformer
  [x] No data leakage (pipeline handles CV)
  [x] Multiple model comparison
  [x] Hyperparameter tuning
  [x] Final test set evaluation (once)
  [x] Metrics: MAE, RMSE, R-squared

This project used concepts from:
  ML preprocessing tutorial: Data Preprocessing
  feature engineering tutorial: Feature Engineering
  train/test split tutorial: Train/Test Split
  model comparison tutorial: Comparing Algorithms
  model evaluation tutorial: Model Evaluation
  hyperparameter tuning tutorial: Hyperparameter Tuning
  ML pipeline tutorial: ML Pipeline

What Could Go Wrong

The moment you point this pipeline at your own data, things break in new ways, the way a recipe that runs perfectly in your own kitchen can still flop on a friend's stove. Here are the three traps that bite people most often, and how to spot each one fast:

• Your numbers do not match mine exactly. Check that the seed line rng = np.random.default_rng(42) runs before you build the DataFrame, and that you did not run any other random call in between. A different seed, or the same seed used in a different order, gives different houses and so different metrics.
• The pipeline crashes on a category it has never seen. If a test row has a neighborhood that never appeared in training, a plain encoder raises an error. We dodge this with OneHotEncoder(handle_unknown="ignore"), which quietly encodes the unknown value as all zeros instead of crashing. Drop that argument and watch it break.
• Your test score looks too good to be true. Nine times out of ten this means a leak: you scaled or imputed using the whole dataset before splitting, so the test rows secretly influenced the training step. Keep every transform inside the Pipeline, the way we did, and the split stays clean.

Extend It

The build works, but a real project is never done. Pick one of these and make it yours. No step-by-step instructions on purpose: figuring out the how is where the learning lives.

• Add a curved feature. Square total_sqft or multiply it by overall_qual, add it as a new column, and see whether the tree models finally beat Ridge.
• Swap in real data. Load the California housing dataset from sklearn.datasets.fetch_california_housing and run the same pipeline. On real, slightly curved data the winner may flip.
• Print the dollar impact of each feature. After fitting Ridge, pull out .coef_ and translate the scaled coefficients back into "this much square footage is worth this many dollars".
• Save and reload the model. Use joblib.dump to save the whole fitted pipeline, then load it in a fresh script and predict the price of one made-up house.

Frequently Asked Questions

Interview Questions on House Price Prediction

Try each one aloud first. The phrasing you produce under mild pressure is what interviews measure.

Q: In an end-to-end regression project, why do you split the data into train and test sets before you scale or impute anything?

Because scaling and imputation learn from the data, and if they see the test rows first, information from the test set leaks into training and inflates your score. The correct order is split first, then fit every transform on the training rows only. In scikit-learn you get this for free by putting the imputer, scaler, and encoder inside a Pipeline, so each cross-validation fold refits the transforms on just that fold's training data.

Q: MAE, RMSE, and R-squared all describe this model. When would you report MAE over RMSE?

MAE is the average dollar error and treats every mistake equally, so it is easy to explain to a non-technical stakeholder ("we are off by about $17,000 on average"). RMSE squares the errors first, so it punishes large misses harder, which matters when a few huge mispredictions are especially costly. Report MAE when all errors hurt about the same and you want an interpretable number, and lean on RMSE when big outlier errors are the real risk.

Q: Scenario: your test-set R-squared is 0.99 but the same model scores 0.72 on live data the next month. What do you check first?

First suspect data leakage: a feature that was available at training time but not at prediction time, or preprocessing that touched the full dataset before the split. Then check for distribution shift, meaning the live houses differ from your training houses (new neighborhoods, a changed price range, seasonality). Confirm the training features are computed the exact same way in production, and validate on a fresh, correctly held-out sample before trusting the drop.

Q: Scenario: in production the pipeline crashes on a row whose neighborhood value never appeared during training. How do you fix it?

That is the classic unseen-category error from the encoder. Set OneHotEncoder(handle_unknown="ignore") so an unknown category is encoded as all zeros instead of raising. For a longer-term fix, monitor category drift and periodically retrain so new neighborhoods become known values, and consider target or ordinal encoding if the category count keeps growing.

Q: Why compare three models with cross-validation instead of just training the fanciest one and reading its test score?

A single train/validation split can be lucky or unlucky, so its score is noisy. Cross-validation averages the score across several folds, giving a more stable estimate of how each model generalizes. You compare all candidates on that averaged score and keep the test set untouched, so the final number stays an honest measure of the winner rather than a figure you optimized against.

Q: The grid search reported that the default alpha of 1.0 was best. Is tuning a waste of time when that happens?

No. Tuning is a search, not a guarantee of improvement, and confirming the default is optimal is itself a useful result backed by cross-validation evidence rather than a guess. It also tells you the model is not very sensitive to that hyperparameter over the range you tried, which is worth knowing. The real payoff is the disciplined process: search on training folds, then grade once on the held-out test set.

Part 5: Machine Learning is Complete

Congratulations, you have finished Part 5: Machine Learning. Across 31 posts (122 to 152), you went from gradient descent intuition all the way to a full house price prediction pipeline. You now know linear and logistic regression, decision trees, random forests, gradient boosting, Support Vector Machines (SVM), K-Nearest Neighbors (KNN), Naive Bayes, clustering, Principal Component Analysis (PCA), model evaluation, hyperparameter tuning, pipelines, anomaly detection, recommender systems, time series, Natural Language Processing (NLP), sentiment analysis, deployment, and best practices.

Next up: a 40-question Machine Learning interview checkpoint to lock in everything from Part 5, before Part 6: Deep Learning and AI, where neural networks, transformers, Large Language Models (LLMs), and AI agents await. Want the full map of where you have been and where you are headed? Browse the complete Python + AI/ML tutorial series home.

Series: Python + AI/ML Cookbook. Part 5: Machine Learning

Go deeper: the official Python documentation covers every edge case of this topic.

Related Posts

Previous: ML: Best Practices, the Ethics and Catches Nobody Warns You About

Next: Machine Learning Interview Questions: 40-Question Checkpoint

Series Home: Python + AI/ML Tutorial Series