Free explainer · From the world of the book
Why do LLMs hallucinate? The honest answer, in plain English
In 2023, a New York lawyer submitted a legal brief citing six court cases. They were well-formatted, plausibly named, and completely fictional: ChatGPT had invented every one of them. The lawyer was sanctioned, the story made global headlines, and the word hallucination entered everyday vocabulary.
Three years on, models are far better, and people still get caught out, because most explanations of hallucination are either too technical or too reassuring. Here's the honest version.
The short answer
That's it. Everything else is detail. But the detail is where the useful intuition lives, so let's build it properly.
An LLM is a prediction machine, not a database
When you ask a search engine a question, it looks things up. When you ask an LLM a question, it does something completely different: it generates an answer one token at a time, choosing each token because it's statistically likely given everything written so far. (If "token" is new to you, start with how LLMs work in plain English.)
Training taught the model the patterns of hundreds of billions of sentences. Ask it the capital of France and "Paris" is overwhelmingly the likeliest continuation, the pattern appears millions of times in its training data, so prediction and truth line up perfectly. This is why LLMs are right so often: for well-documented facts, the most plausible answer is the true answer.
The trouble starts at the edges of the model's knowledge. Ask about an obscure academic paper, a small company's founding date, or a court case that doesn't exist, and the model can't say "no pattern found" the way a database returns zero rows. Its machinery does the only thing it can do: produce the most plausible-looking continuation. Plausible citation format + plausible author names + plausible year = a reference that has never existed.
Why does it sound so confident?
Here's the part that trips up even careful people. The confident tone is also predicted text. Human writing in the training data rarely says "I'm about 60% sure the capital is Paris", people either answer or don't. So the model learned that answers come wrapped in confident prose, and it wraps everything in confident prose, including its fabrications.
For an LLM, sounding certain and being right are two unrelated properties. That single sentence, properly absorbed, will save you from most hallucination-related mistakes.
Four reasons hallucination is built in
1. The training objective rewards plausibility
Models are trained to predict the next token accurately across their training data. Nothing in that objective checks statements against reality. Truth correlates with plausibility for common knowledge, and diverges for rare, recent or invented topics.
2. Benchmarks and training have rewarded guessing
Research (including OpenAI's own 2025 analysis) found that common evaluation methods score a confident wrong answer the same as "I don't know", or better. Like a student who never leaves a multiple-choice question blank, models learned that guessing pays. Newer training approaches now reward calibrated uncertainty, which is one reason recent models decline to answer more often.
3. Knowledge is compressed, not stored
Training compresses an internet's worth of text into model weights. Compression loses detail, especially rare detail. The model retains the shape of a citation, a biography, an API, even where it has lost (or never had) the specifics. Hallucination is that shape, filled in with statistically plausible filler.
4. It can't perceive its own knowledge gaps
A database knows when a record is missing. An LLM has no such lookup: every answer, real or fabricated, is produced by the same smooth prediction process. From the inside, inventing a fact feels no different from recalling one.
What hallucinations look like in the wild
- Fabricated citations and quotes: real-sounding papers, cases and quotations that don't exist. The classic failure in law, academia and journalism.
- Invented specifics: dates, statistics, prices and version numbers that are plausible but wrong.
- Non-existent functions and packages: code assistants confidently importing libraries that were never written (a real security risk: attackers register the hallucinated package names).
- Playing along with false premises: ask "why did Company X go bankrupt in 2019?" about a healthy company and the model may write you a convincing post-mortem of an event that never happened.
- Confabulated memories: claims about earlier parts of your conversation that fell outside the model's context window.
Can hallucination be fixed?
Reduced substantially, not eliminated. The main levers:
- Retrieval-augmented generation (RAG): give the model real documents or live search results to ground its answer. This helps a lot, but models can still misread or over-generalise what they retrieve.
- Better incentives: training that rewards "I don't know" over confident guessing.
- Citation and verification layers: systems that check each generated claim against sources before showing it to you.
All three make hallucination rarer. None changes the underlying fact that the engine generates plausible text. Treat "hallucination-free AI" claims the way you'd treat "unsinkable ship".
How to protect yourself (and your team)
- Be most suspicious of specifics. Names, numbers, dates, citations, URLs and quotes are where hallucinations concentrate. Generic explanations are safer ground.
- Verify anything you'll act on against a source the model didn't write. This is the one habit that matters most.
- Don't ask the model if it's sure. "Are you sure?" just prompts more plausible text, sometimes a confident retraction of a correct answer.
- Use grounded modes for factual work. Prefer tools that cite sources you can click and check.
- Match the stakes. Brainstorming and drafting tolerate hallucination; legal, medical, financial and reputational work does not.
Test yourself
Think this is obvious now? Our 2-minute AI knowledge quiz catches most people on at least three questions, hallucination included.
This article is the book in miniature
How LLMs Actually Work builds the full picture of tokens, training, attention, hallucination and practical judgement, in plain English, cover to cover in a weekend.