“What if it just makes something up?” is usually the first question a risk committee asks about any LLM deployment — and it deserves a measured answer, not a reassurance.
Hallucination isn’t one failure mode — it covers several distinct patterns: factual…
Groundedness scoring — for RAG systems specifically, check whether each claim in the output…
Grounding the model in retrieved, verifiable source documents (RAG) reduces fabrication…
Hallucination isn’t one failure mode — it covers several distinct patterns: factual fabrication (stating something false as true), source misattribution (citing a real source for a claim it doesn’t support), unsupported extrapolation (going beyond what the available context actually establishes), and confident uncertainty (presenting a genuine guess with the same confidence as a verified fact). Each needs a different test design, since they fail in different ways and respond to different mitigations.
Grounding the model in retrieved, verifiable source documents (RAG) reduces fabrication substantially compared to relying on parametric knowledge alone, because the model has an actual source to draw from rather than generating from statistical patterns in training data. Explicit instructions to express uncertainty or decline to answer when the context doesn’t support a confident response measurably reduce confident-but-wrong outputs. Structured output formats that require citing a specific source passage for each claim make unsupported fabrication both less likely and easier to catch in review.
A hallucination in an internal brainstorming tool is a minor annoyance. A hallucination in a customer-facing financial or medical information system is a serious incident. Hallucination testing rigor, acceptable-rate thresholds, and required human review should scale with how consequential a wrong answer actually is — not be applied uniformly across every LLM feature regardless of stakes.
Not reliably with current LLM technology. The practical goal is measuring and reducing the rate to an acceptable level for the specific use case, combined with appropriate human review or verification steps for higher-stakes outputs, rather than expecting zero.
It substantially reduces fabrication of facts not in the source material, but doesn’t eliminate it — a model can still misread, misattribute, or over-extrapolate from retrieved documents. RAG reduces one major cause of hallucination, not every cause.
Ideally as part of continuous regression evaluation triggered by any change to the prompt, model version, or retrieval pipeline — hallucination rate can shift silently with any of these changes, the same way other quality metrics can degrade without a visible signal otherwise.