Transfer Learning
Reusing a model trained on one task as the starting point for a different task, so new problems can be solved with far less data and compute.
Transfer learning is the idea that knowledge gained on one problem can be repurposed for another. A vision model trained to recognise everyday objects already encodes edges, textures, and shapes; adapting it to spot tumours needs far less data than training from scratch.
The pattern usually has two stages:
- Pre-training: learn general representations from large, broad data.
- Adaptation: fine-tune the model on a smaller, task-specific dataset — sometimes by updating all weights, sometimes by training a small adapter or only the final layers.
Transfer learning is the foundation of the modern AI stack. Foundation models are extreme examples: one pre-trained model is adapted to translation, coding, customer support, and hundreds of other tasks. It also drives few-shot learning — a model generalises from a handful of examples because it already understands the underlying structure of the problem.
Limits exist: if the source and target domains are too different, or the fine-tuning data is biased, transferred knowledge can hurt rather than help — a phenomenon called negative transfer.