Recurrent Neural Network RNN

A network that processes sequences one element at a time, carrying a hidden state forward — the standard approach before transformers.

Deep Learning Advanced 1 min read

Also known as: RNN

Definition

An RNN reads a sequence step by step, maintaining a hidden state that carries information forward. That makes it a natural fit for text and time series, and it was the dominant sequence architecture for years.

Two problems ended that. Gradients vanish or explode over long sequences, so plain RNNs struggle to connect distant events — LSTMs and GRUs were built to mitigate this. And the sequential dependency makes training fundamentally unparallelisable, which is what the transformer removed.

How it relates to other terms

Contrast with Transformer The neural network architecture behind almost all modern language models, built around attention instead of re... Broader term Neural Network A model built from layers of simple weighted units, trained by adjusting those weights until the output matche...
Referenced by
Transformer Contrast with

Related terms

Attention

A mechanism that lets a model weigh how much each part of the input should influence each output, learned rather than fixed.

Intermediate Deep Learning

Transformer

The neural network architecture behind almost all modern language models, built around attention instead of recurrence.

Intermediate Deep Learning

Backpropagation

The algorithm that works out how much each weight in a network contributed to the error, by applying the chain rule backwards through the layers.

Advanced Deep Learning

Gradient Descent

The optimisation method that trains most models: repeatedly step every parameter a little way downhill on the error surface.

Intermediate Deep Learning

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction