Transfer Learning

Reusing a model trained on one task as the starting point for another, instead of training from scratch.

Machine Learning Intermediate 1 min read

Definition

Transfer learning takes the representations a model learned on a large general task and reuses them for a narrower one. The early layers of a vision model learn edges and textures that are useful for almost any image task; a language model's representations of syntax and meaning transfer similarly.

It is the reason a useful classifier can be built from a few hundred labelled examples rather than a few hundred thousand, and the conceptual foundation under both fine-tuning and the whole pre-train-then-adapt paradigm.

How it relates to other terms

Narrower term Fine-Tuning Continuing to train an existing model on a smaller, specific dataset so it adapts to a particular task, domain... See also Supervised Learning Training on examples that come with the right answer attached, so the model learns to map inputs to known labe...
Referenced by
Fine-Tuning Narrower term

Related terms

Overfitting

When a model learns the training data so closely — including its noise — that it performs worse on anything new.

Beginner Machine Learning

Embedding

A list of numbers representing a piece of data, arranged so that similar things end up close together.

Intermediate Machine Learning

Neural Network

A model built from layers of simple weighted units, trained by adjusting those weights until the output matches the target.

Beginner Machine Learning

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction