Most RAG Hallucinations Are Retrieval Failures: How the Retrieval Brick Decides What the Model Can Invent

Most so-called AI hallucinations in Retrieval-Augmented Generation (RAG) systems are not the model inventing facts, but rather retrieval failures that feed it the wrong information. Standard retrieval methods often fail because they match the general meaning of a query, causing them to overlook the single document with the precise answer in favor of topically related but incorrect ones. This results in the model either receiving no correct context, receiving the wrong context, or having the right answer buried in a sea of irrelevant information. In a striking example with a cybersecurity document, a query about "backup practices" caused a standard system to rank the only page mentioning "backup" as the least relevant of all. Fixing the retrieval step is therefore the key to preventing most of what are logged as model hallucinations.