Overfitting

When a model learns the training data so closely — including its noise — that it performs worse on anything new.

Machine Learning Beginner 1 min read Start here

Definition

An overfitted model has memorised rather than generalised. The tell is a widening gap between training performance and held-out performance: training loss keeps falling while validation loss flattens and then rises.

The usual causes are too little data for the model's capacity, or training for too long. The usual remedies are more data, augmentation, regularisation, dropout, and stopping early.

Its opposite, underfitting, is a model too simple to capture the real pattern — bad on training data and test data alike.

Why it matters

Almost every impressive-looking result that fails in production failed here first. It is the reason a held-out test set is not optional.

How it relates to other terms

See also Gradient Descent The optimisation method that trains most models: repeatedly step every parameter a little way downhill on the... See also Supervised Learning Training on examples that come with the right answer attached, so the model learns to map inputs to known labe...
Referenced by
Gradient Descent See also Supervised Learning See also

Related terms

Embedding

A list of numbers representing a piece of data, arranged so that similar things end up close together.

Intermediate Machine Learning

Neural Network

A model built from layers of simple weighted units, trained by adjusting those weights until the output matches the target.

Beginner Machine Learning

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction