Hallucination

Language models generate the most plausible continuation of text, not a verified fact. When the right information is missing from the prompt or the model's training, it can still produce an answer that reads well, such as a citation that does not exist, a function that is not in the library, or a detail that contradicts the provided document. That is a hallucination, sometimes called confabulation.

In agents the stakes are higher than in chat: a made-up file path, API parameter, or tool result can propagate through later steps, or trigger a real action. Hallucination is therefore treated as a reliability and safety issue, not just a quality quirk.

It cannot be eliminated outright, but it can be reduced and contained.

How it works

Common mitigations work in layers. Grounding: give the model the relevant source material through retrieval (agentic-rag) or tools, and require answers to be based on it. Constraining: restrict outputs to declared options or schemas so invalid answers are impossible, though a valid-looking wrong answer still is (see jev). Verifying: run tests, compile code, or check claims against sources before acting. Escalating: let the model say it is unsure and route those cases to a person (human-in-the-loop). Measuring: track hallucination rates in agent-evaluation.

Example

A coding agent is asked to call a library function and confidently uses a method that does not exist. A compile or test step catches the error and feeds the failure back so the agent corrects it. Without that check, the fabricated call would have been committed.

How it differs

Common misconceptions

FAQ