Hallucination

When a model states something false with the same fluency and confidence it uses for things that are true.

Ethics & Safety Beginner 1 min read Start here

Also known as: Confabulation

Definition

A hallucination is model output that is fluent, plausible and wrong. Invented citations, non-existent API methods and confidently misremembered dates are the classic examples.

It is not a bug in the usual sense. A language model is trained to produce likely-looking text, and a fabricated but well-formed citation is extremely likely-looking. The model has no separate representation of "things I actually know" to check against.

Retrieval, tool use and asking for sources all reduce the rate. None of them eliminate it, and the term itself is contested — some researchers prefer "confabulation", on the grounds that "hallucination" implies a perceptual failure the model is not having.

Why it matters

It is the single most important limitation to understand before putting a model anywhere its output is not checked.

How it relates to other terms

See also Retrieval-Augmented Generation Fetching relevant documents at query time and putting them in the prompt, so the model answers from your data... See also Temperature A setting that controls how much randomness goes into choosing each token — low is predictable, high is varied... See also Alignment The problem of getting a system to pursue what we actually intend, rather than a proxy that merely correlates...
Referenced by
Large Language Model See also Retrieval-Augmented Generation Contrast with Temperature See also Alignment See also Chain-of-Thought See also

Related terms

RLHF

Tuning a model using human preference comparisons, so it produces the kind of answer people actually rate highly.

Advanced Ethics & Safety

Alignment

The problem of getting a system to pursue what we actually intend, rather than a proxy that merely correlates with it.

Intermediate Ethics & Safety

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction