Large Language Model LLM

A neural network trained on very large amounts of text to predict what comes next, which turns out to be enough to answer questions, write code and hold a conversation.

Natural Language Beginner 1 min read Start here

Also known as: LLM

Definition

A large language model is a neural network — almost always a transformer — trained on a very large corpus of text with a deceptively simple objective: given some text, predict the next token.

Nothing about that objective mentions reasoning, translation or code. Those capabilities emerge because doing next-token prediction well across a broad enough corpus requires modelling a great deal about how the world is described. Scaling the model, the data and the compute together tends to improve those emergent capabilities, which is the observation the last several years of the field have been built on.

An LLM has no memory between conversations unless one is bolted on, and no access to information outside its training data unless it is given tools or retrieval. Both of those limits are engineering problems rather than properties of the model.

Key points

Trained to predict the next token, nothing more
Capabilities emerge from scale rather than being programmed
Stateless between calls unless memory is added
Knows nothing past its training cutoff without retrieval

Why it matters

Nearly every consumer-facing AI product released since 2022 is an LLM with a interface around it. Understanding what the model does and does not do explains most of what those products get right and wrong.

How it relates to other terms

Prerequisite Transformer The neural network architecture behind almost all modern language models, built around attention instead of re... Prerequisite Token The unit a language model actually reads and writes — usually a word fragment rather than a word or a characte... See also Hallucination When a model states something false with the same fluency and confidence it uses for things that are true. See also Context Window The maximum number of tokens a model can consider at once — everything it is allowed to "see" for a single res...
Referenced by
Transformer Broader term Token Includes RLHF See also

Related terms

Token

The unit a language model actually reads and writes — usually a word fragment rather than a word or a character.

Beginner Natural Language

Context Window

The maximum number of tokens a model can consider at once — everything it is allowed to "see" for a single response.

Beginner Natural Language

Chain-of-Thought

CoT

Prompting a model to work through its reasoning step by step before answering, which measurably improves multi-step problems.

Intermediate Natural Language

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction