Transfer learning lets you reuse a model that already learned to see, then point it at your own problem with a tiny dataset. This transfer learning Python guide shows feature extraction versus fine-tuning with ResNet, VGG, and EfficientNet, so you can classify custom images with far less data than training a network from scratch.

“After supervised learning, transfer learning will be the next driver of ML commercial success.”

Andrew Ng, NIPS 2016 tutorial

Last Updated: July 2026 | Tested on: Python 3.14.6, PyTorch 2.12.1, torchvision 0.27.1 | Difficulty: Advanced | Reading Time: 9 minutes

Training a Convolutional Neural Network (CNN) from scratch needs a mountain of data. ImageNet has 1.2 million images across 1,000 classes, and the best models trained on it for days across many Graphics Processing Units (GPUs). Most real problems have a few hundred or a few thousand images, not millions. Transfer learning fixes this by starting from a model that already knows how to see. It has learned edges, textures, patterns, shapes, and whole objects. You keep all of that and only swap the final layer for your own classes, then tune it on your small dataset.

Think of it like hiring a chef who already knows how to cook. You do not teach them how to hold a knife or boil water. You just hand them your recipe and they pick it up in an afternoon. A model pre-trained on ImageNet is that experienced chef. Your small dataset is the new recipe. The years of basic skill are already baked in, so you only train the last little bit.

You get two ways to do this. Feature extraction freezes every pre-trained layer and trains only the new classifier head on top. It is fast (minutes, not hours) and works great when your images look like ImageNet, that is, ordinary photos. Fine-tuning unfreezes some or all of the pre-trained layers and nudges them with a very small learning rate. You reach for this when your domain is different from ImageNet (medical scans, satellite photos, factory defect images) and the borrowed features need a little adjusting.

Here is what we cover:

• Feature extraction: freezing a pre-trained model and training a new head
• Fine-tuning: unfreezing layers so the model adapts to your domain
• Using ResNet, VGG, and EfficientNet from torchvision
• A complete transfer learning pipeline for custom image classification

Prerequisites

Transfer Learning Architecture

The diagram walks through the transfer learning workflow using a ResNet as the example. You take a model trained on millions of images, freeze its early layers that spot general features (edges, then textures, then parts), keep them as a fixed feature extractor, and bolt a fresh classifier on the end for your own classes. Only the new head, and optionally the last block or two, actually learn. The early layers already understand what an edge or a texture looks like, so your model only needs to learn how those features map to your labels.

That is why a few hundred labeled images can be enough to train an accurate classifier. The diagram names ResNet-50 as a stand-in, but the same picture holds for the ResNet-18 we use in the code below.

Feature Extraction with ResNet

Feature extraction is like hiring a seasoned wildlife photographer and asking them to sort your holiday photos into albums. Their trained eye for shapes, light, and texture stays exactly as it is; you only teach them your particular set of labels. In code, that means freezing every pre-trained layer and training just a small new head on top.

📄 transfer_feature_extraction.py: using ResNet as a feature extractor

import torch
import torch.nn as nn
from torchvision import models

torch.manual_seed(42)  # so the random head and forward pass are reproducible

# Vinay loads a pre-trained ResNet-18
resnet = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)

# Freeze all parameters
for param in resnet.parameters():
    param.requires_grad = False

# Replace the final classification layer
num_features = resnet.fc.in_features  # 512 for ResNet-18
resnet.fc = nn.Sequential(
    nn.Linear(num_features, 128),
    nn.ReLU(),
    nn.Dropout(0.3),
    nn.Linear(128, 5),  # 5 custom classes
)

# Count parameters
total = sum(p.numel() for p in resnet.parameters())
trainable = sum(p.numel() for p in resnet.parameters() if p.requires_grad)
print(f"Total parameters:     {total:>10,}")
print(f"Trainable parameters: {trainable:>10,}")
print(f"Frozen parameters:    {total - trainable:>10,}")
print(f"Trainable ratio:      {trainable/total:.2%}")

# Test forward pass
x = torch.randn(4, 3, 224, 224)  # 4 images, 3 channels, 224x224
output = resnet(x)
print(f"\nInput shape:  {x.shape}")
print(f"Output shape: {output.shape}")
print(f"Predictions:  {output.argmax(dim=1).tolist()}")

▶ Output

Total parameters:     11,242,821
Trainable parameters:     66,309
Frozen parameters:    11,176,512
Trainable ratio:      0.59%

Input shape:  torch.Size([4, 3, 224, 224])
Output shape: torch.Size([4, 5])
Predictions:  [0, 4, 2, 4]

What happened here: ResNet-18 ships with about 11.2 million parameters already trained on ImageNet. We froze every one of them by setting requires_grad = False, then replaced the final fc layer with our own small head that ends in 5 outputs, one per class. That head adds just 66,309 parameters, which is only 0.59% of the model. The frozen layers do the heavy lifting of turning pixels into rich features, and our tiny head just learns to map those features to our 5 classes.

Because only 0.59% of the weights ever change during training, a fit that would take hours from scratch finishes in minutes. The torch.manual_seed(42) at the top is what makes the random head weights and the random test batch come out the same on every run, so your numbers will match the output above. The predictions themselves are meaningless here (the head has not been trained yet, and the input is random noise); we print them only to confirm the forward pass produces one score per class for each of the 4 images.

Common Mistakes

Interview Corner

Q: When should I fine-tune vs use feature extraction?

Reach for feature extraction when your dataset is small (under 1,000 images) and your pictures look like ImageNet, that is, ordinary photos. Reach for fine-tuning when your domain is genuinely different (medical images, aerial photos, microscopy) or you have enough data (say 5,000 images and up) to adjust the borrowed features without overfitting. The safe move in any transfer learning project: start with feature extraction as your baseline, see how far it gets you, then fine-tune only if you need more accuracy.

Practice Exercises

1. Use feature extraction with ResNet-50 and EfficientNet-B0 on the same dataset. Compare accuracy.
2. Implement gradual unfreezing: feature extraction for 5 epochs, then unfreeze last 2 layers, then unfreeze all.
3. Compare training time and accuracy for training from scratch vs transfer learning on a 500-image dataset.

More in this series:

• DL: Optimizers, Why Your Model Learns (or Doesn’t)
• DL: RNNs and LSTMs for Sequence Processing
• NLP: Word Embeddings (Word2Vec, GloVe, FastText)

Frequently Asked Questions

Interview Questions on Transfer Learning

The same ideas as they show up in real interviews, framed as scenarios you can practice out loud.

Q: In the feature-extraction example only about 0.59% of the parameters are trainable. Why is that a good thing?

The 11.2 million frozen ImageNet weights already convert raw pixels into rich features, so only the small 66k-parameter head has to learn how those features map to your classes. Fewer trainable parameters means far less risk of overfitting a small dataset and much faster training, minutes instead of hours. You keep the representational power of a large model without paying its full training cost.

Q: What does setting requires_grad = False actually do?

It tells PyTorch to skip gradient computation and weight updates for those parameters during backpropagation. That freezes the features the model already learned on ImageNet and removes them from the optimizer’s work, which is exactly why the frozen backbone acts as a fixed feature extractor. Only the parameters left with requires_grad = True (your new head) will change during training.

Q: You fine-tune a pre-trained ResNet and validation accuracy collapses far below your feature-extraction baseline. What do you check first?

The usual culprit is too large a learning rate on the pre-trained layers, which wipes out the borrowed features in the first few batches. Drop the LR to something like 1e-5, roughly 10x to 100x smaller than the head’s 1e-3, or freeze the early layers and unfreeze only the last block. Also confirm you are applying the same ImageNet mean/std normalization the model was pre-trained with, since a preprocessing mismatch produces the same symptom.

Q: Your inputs are 3-channel medical scans, the model trains without errors, but validation accuracy never improves. What might be wrong?

Check preprocessing first: pre-trained models expect the exact ImageNet normalization and 224×224 input, and a silent mismatch flattens accuracy. Because medical scans look nothing like ImageNet photos, pure feature extraction may not transfer well, so unfreeze the deeper layers and fine-tune with a small learning rate. Finally, verify your labels are correct and that the final layer’s output size matches your number of classes.

Q: Why does the new classifier head start with random weights while the rest of the network is pre-trained?

The original final layer predicted ImageNet’s 1,000 classes, which do not match your task, so you replace it with a fresh layer sized to your own classes. That new layer has never been trained, so its weights start random and get learned during training. The pre-trained body keeps its weights and feeds strong features into your new head, which is why the head converges quickly.

Q: How much data do you need for transfer learning versus training a CNN from scratch?

Training from scratch usually needs tens or hundreds of thousands of images to generalize well. With feature extraction you can often get strong results from a few hundred images, sometimes just 20 to 50 per class when the classes look clearly different. Fine-tuning sits in between and typically wants 500 or more images per class to adjust the borrowed features without overfitting.

What’s Next?

You now know the two core moves of transfer learning: feature extraction, where you freeze a pre-trained backbone and train only a fresh head, and fine-tuning, where you unfreeze layers and nudge them with a tiny learning rate to adapt to your own domain. You saw how ResNet-18 turns 11.2 million ImageNet weights into a ready-made feature extractor, so a 66k-parameter head can learn your classes from just a few hundred images instead of millions.

CNNs handle spatial data (images). In the RNN and LSTM tutorial, we tackle sequential data (text, time series, and anything where order matters) using recurrent neural networks and their improved variant, Long Short-Term Memory networks (LSTMs).

Want the full path from Python basics to deep learning? Browse the complete Python + AI/ML tutorial series home for every lesson in order.

Go deeper: when you outgrow this post, the official Python documentation is the next stop.

Related Posts

Previous: DL: CNNs, Convolution, Pooling, Image Classification

Next: AI: Computer Vision Project, Object Detection with YOLO

Series Home: Python + AI/ML Tutorial Series