Large Language Models (LLMs) have demonstrated a remarkable ability to generate fluent, coherent, and human-like text. However, beneath this polished exterior lies a significant challenge: hallucination. This phenomenon, where an LLM generates information that is nonsensical, factually incorrect, or unfaithful to a provided source, is one of the most critical hurdles to their reliable deployment. This article explores what hallucinations are, how our understanding of them has evolved, and the methods being developed to detect them.
- A hallucination is model output that is plausible but nonfactual, or inconsistent with the context it was given. Research separates factuality hallucinations from faithfulness hallucinations.
- Hallucination detection started in neural machine translation, with coverage penalties, attention analysis, and measures of how much the output relies on the source.
- Black-box methods such as SelfCheckGPT, semantic entropy, and MetaQA check whether several responses, or mutated versions of one, stay consistent. They need no access to the model's internals.
- White-box methods read token probabilities, logit entropy, or hidden states, so they need an open model or an API that exposes log-probabilities.
- No single method catches every hallucination. Combining scores, and treating detection as one layer of defense, covers more failures.
What is hallucination?
Hallucination in LLMs refers to the generation of content that is plausible yet nonfactual or inconsistent with the provided context. The field has evolved to distinguish between two primary types: factuality hallucination (discrepancy with verifiable real-world facts) and faithfulness hallucination (divergence from user input or lack of self-consistency).
Prompt: "Who was the first person to walk on Mars?"
Hallucinated response: "The first person to walk on Mars was cosmonaut Alexei Leonov in 1989, as part of the secret Soviet 'Ares-1' mission."
Reality: Humans have not yet walked on Mars. The answer is a complete fabrication, but it uses real names (Alexei Leonov was a famous cosmonaut) and plausible details to sound convincing.
Evolution of hallucination types
Historical categories (pre-LLM era)
The term "hallucination" in AI dates back to 1995, when Stephen Thaler demonstrated how artificial neural networks produce phantom experiences through random perturbations. In the early 2000s, it had a positive connotation in computer vision for image enhancement. The late 2010s marked a semantic shift when Google researchers used it to describe neural machine translation (NMT) models generating outputs unrelated to source text.
Modern LLM era classification
Current research categorizes hallucinations into:
- Factuality hallucination - further divided into factual inconsistency and factual fabrication.
- Faithfulness hallucination - subdivided into instruction inconsistency, context inconsistency, and logical inconsistency.
Two fundamental distinctions have emerged: intrinsic vs. extrinsic hallucinations (based on relationship to input context), and factuality vs. faithfulness (based on absolute correctness vs. adherence to input).
Historical evolution of detection methods
Early methods (Neural Machine Translation era)
The first systematic studies of hallucinations appeared in neural machine translation (NMT). Researchers noticed that models sometimes produced fluent but unfaithful translations, and proposed diagnostics such as coverage penalties (ensuring each source token was translated) and alignment consistency checks. Hypotheses like low source contribution (the model ignoring much of the input) and local source contribution (over-reliance on only a few input tokens) were explored. Early interpretability techniques, including attention analysis and later gradient-based methods like Layerwise Relevance Propagation (LRP), were also applied to study these errors. Detection is one layer of a wider defense; the Swiss cheese model shows how several imperfect layers combine to catch what any single check misses.
Pre-LLM detection approaches
Before the rise of large-scale generative LLMs, detection relied on heuristics and interpretability: analyzing attention distributions in RNN and Transformer models, measuring relative source–target contributions, applying coverage penalties, and examining whether model confidence correlated with input alignment. These approaches aimed to flag outputs that "looked" fluent but showed weak grounding in the source.
Current LLM detection methods
Black-box (closed-source) methods
SelfCheckGPT represents a breakthrough black-box approach that leverages sampling-based consistency checking. If an LLM has knowledge of a concept, sampled responses are similar and contain consistent facts. For hallucinated facts, stochastically sampled responses diverge and contradict each other.
Modern black-box methods include consistency-based scorers like non-contradiction probability (whether multiple responses avoid contradicting each other), normalized semantic negentropy (uncertainty between response pairs), normalized cosine similarity, BERTScore, and BLEURT. Discrete semantic entropy represents a particularly sophisticated approach that clusters semantically equivalent answers before computing uncertainty, addressing the limitation that identical meanings can be expressed in different ways. These methods use one or more LLMs to evaluate reliability without requiring access to internal model parameters.
White-box (open-source) methods
Recent advances include MIND (an unsupervised training framework leveraging internal states for real-time detection), and approaches analyzing internal hidden states, attention maps, and output prediction probabilities. Semantic-entropy-based methods detect confabulations by computing uncertainty at the meaning level rather than specific word sequences, addressing the fact that one idea can be expressed multiple ways.
White-box methods leverage token probabilities, including minimum token probability, length-normalized token probability, logit entropy scores, and windowed logit entropy for more refined detection.
Advanced methods
MetaQA (Yang et al., 2025) uses metamorphic relations and prompt-mutation to detect hallucinations without relying on external knowledge sources. It works by generating mutated variants of the model's response or prompt (e.g. using synonym/antonym transformations) and checking if semantic consistency holds. Violations of these relations are taken as evidence of hallucination. MetaQA is compatible with both open- and closed-source LLMs and outperforms SelfCheckGPT in precision, recall, and F1 across multiple benchmarks.
Recent work explores token-level Entropy Production Rate (EPR) metrics as lightweight uncertainty estimators. While they do not require internal states (hence "black-box"), they assume access to token log-probabilities - available in some API settings but not universally exposed.
Comparing the detection methods
The methods above differ mainly in what they need from the model. The table compares them side by side.
| Method | What it needs | How it detects | Limitations |
|---|---|---|---|
| SelfCheckGPT | Black-box: several sampled responses to the same prompt. No model internals. | Checks whether the samples agree. Facts the model knows stay consistent; hallucinated facts diverge and contradict each other. | Each check costs several extra generations. |
| Semantic entropy | Several sampled answers. The discrete variant works without token probabilities. | Clusters answers that share a meaning, then measures uncertainty across the clusters. High entropy signals confabulation. | Needs repeated sampling, which costs more than a single answer. |
| Token-probability thresholds | White-box: token probabilities or logits from the model. | Flags low minimum token probability, low length-normalized probability, or high logit entropy. | Closed APIs do not always expose token probabilities. |
| Entropy Production Rate (EPR) | Token log-probabilities from a single response, such as the top candidates an API returns. | Uses token-level entropy as a lightweight uncertainty estimate. | Assumes log-probabilities are available, which not every API provides. |
| MIND | White-box: access to the model's internal states. | An unsupervised framework trained on internal states to flag hallucinations in real time. | Only works with open models whose internals you can read. |
| MetaQA | Black-box: works with open and closed models, and needs no external knowledge source. | Mutates the prompt or response with synonym and antonym relations, then checks whether meaning stays consistent. | Generating and verifying mutations takes many extra model calls per question. |
Key detection frameworks
Current detection strategies fall into two main categories:
Factuality hallucination detection
Fact-checking against trusted knowledge sources and uncertainty estimation via internal signals.
- Black-box: SelfCheckGPT, semantic entropy via output sampling.
- White-box: token probability thresholds, logit entropy, hidden-state analysis.
Faithfulness hallucination detection
Evaluating output faithfulness to contextual information.
- Black-box: consistency scoring, entailment checks, multiple outputs.
- White-box: attention alignment, hidden-state context attribution.
Recent frameworks like multiple-testing approaches combine scores from various methods systematically, leveraging advantages of preexisting methods without additional assumptions about specific datasets or LLMs.
Conclusion
The battle against LLM hallucinations is an ongoing and dynamic field. Our understanding has matured from viewing them as simple "errors" to classifying them with a nuanced taxonomy like intrinsic vs. extrinsic. Similarly, our detection toolkit has expanded from rudimentary word-matching to leveraging LLMs themselves as judges and even peering into the very neurons of open-source models to catch a hallucination before it happens. While perfect detection remains elusive, these advancements are paving the way for more reliable and trustworthy AI systems.
FAQ
What is an AI hallucination?
An AI hallucination is content generated by a large language model that is plausible yet nonfactual, or inconsistent with the context it was given. It can be nonsensical, factually incorrect, or unfaithful to a provided source, and it often sounds convincing because it mixes real names and plausible details.
What is the difference between factuality and faithfulness hallucinations?
A factuality hallucination conflicts with verifiable real-world facts, either by contradicting them or by fabricating them. A faithfulness hallucination diverges from the user's input or context, or is not consistent with itself. It covers instruction inconsistency, context inconsistency, and logical inconsistency.
How can you detect hallucinations without access to the model's internals?
Black-box methods work from the model's outputs alone. SelfCheckGPT samples several responses and checks whether they agree. Semantic entropy clusters answers with the same meaning and measures how much they differ. MetaQA mutates the prompt or response and checks whether the meaning stays consistent.
How does SelfCheckGPT detect hallucinations?
SelfCheckGPT samples several responses to the same prompt and compares them. If the model knows a fact, the sampled responses tend to be similar and consistent. If a fact is hallucinated, the samples tend to diverge and contradict each other.
What is semantic entropy in hallucination detection?
Semantic entropy measures uncertainty at the level of meaning rather than exact word sequences. It groups sampled answers that mean the same thing before computing uncertainty, so the same idea phrased in different ways is not counted as disagreement. High semantic entropy is a sign of confabulation.
Can hallucinations be detected reliably?
Not perfectly. Detection methods have improved from coverage penalties in machine translation to sampling-based checks and analysis of a model's internal states, but perfect detection remains elusive. Detection works best as one layer among several controls.
Resources
Internal resources
- The Swiss cheese model for AI safety
- LLM evaluation guide
- LLM-as-a-judge evaluation
- LLM monitoring
- Evaluating AI agents
- GenGuardX governance platform
External resources
- Huang et al., A Survey on Hallucination in Large Language Models (2023)
- Lee et al., Hallucinations in Neural Machine Translation (Google, 2018)
- Tu et al., Modeling Coverage for Neural Machine Translation (2016)
- Voita et al., Analyzing the Source and Target Contributions to Predictions in NMT (ACL 2021)
- Xu et al., Understanding and Detecting Hallucinations in NMT via Model Introspection (TACL 2023)
- Manakul et al., SelfCheckGPT (EMNLP 2023)
- Farquhar et al., Detecting hallucinations in large language models using semantic entropy (Nature, 2024)
- Bouchard and Chauhan, Uncertainty Quantification for Language Models (2025)
- Su et al., MIND: Unsupervised Real-Time Hallucination Detection (2024)
- Yang et al., Hallucination Detection in LLMs with Metamorphic Relations (2025)
- Moslonka et al., Hallucination Detection using Token-level Entropy Production Rate (2025)
- Li et al., Principled Detection of Hallucinations in LLMs via Multiple Testing (2025)