Beginner · Generative AI
Hallucination
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. AI hallucination is when a model generates confident but factually incorrect or entirely fabricated information, presenting it as truth.
Intuitive Explanation
Imagine asking a very confident friend a question they don't know the answer to. Instead of saying 'I don't know,' they make up a plausible-sounding answer with complete conviction. AI hallucination is exactly this — the model generates text that sounds authoritative but is completely made up.
Technical Definition
Hallucination in language models refers to generated content that is not grounded in the input context or factual knowledge. It arises from the model's training objective (next-token prediction) which optimizes for fluency and plausibility rather than factual accuracy, combined with the model's inability to distinguish between reliable and unreliable training data.
How it works
Hallucination is arguably the biggest unsolved challenge in deploying LLMs for high-stakes applications. **Types of Hallucination:** 1. **Intrinsic:** Contradicts the source/input (e.g., summarization that misquotes the original) 2. **Extrinsic:** Adds information not present in any source (e.g., fabricating citations) 3. **Factual:** States incorrect facts with confidence (e.g., wrong dates, invented events) **Root Causes:** - **Training Objective:** Next-token prediction rewards fluency, not accuracy - **Knowledge Gaps:** The model fills gaps by pattern-matching rather than admitting ignorance - **Data Quality:** Training data contains errors, contradictions, and outdated information - **Decoding Strategy:** High temperature and nucleus sampling increase creativity but also hallucination - **Out of Distribution:** Questions outside training data distribution trigger confabulation **Mitigation Strategies:** 1. **RAG:** Ground responses in retrieved, verifiable documents 2. **Citation Training:** Train models to cite sources for claims 3. **Uncertainty Quantification:** Detect when the model is uncertain and abstain 4. **Constitutional AI:** Train models to refuse rather than fabricate 5. **Fact-Checking Pipelines:** Post-generation verification against knowledge bases 6. **Human-in-the-Loop:** Expert review for critical applications 7. **Constrained Decoding:** Limit generation to factual patterns
Real-World Use Cases
- Google: Bard/Gemini integrates Search to ground responses and reduce hallucination
- Anthropic: Claude is trained with Constitutional AI to refuse rather than fabricate
- Legal AI: Hallucinated legal citations have led to court sanctions (Mata v. Avianca)
- Medical AI: Hallucination in medical AI could lead to dangerous misdiagnoses
Related Concepts
- Large Language Model (LLM) — A massive neural network trained on vast text corpora to understand and generate human language with remarkable fluency.
- Fine-Tuning — Adapting a pre-trained model to a specific task by continuing training on a smaller, task-specific dataset.
- Retrieval-Augmented Generation (RAG) — A technique that enhances LLM responses by retrieving relevant documents from an external knowledge base before generating an answer.