RAG that retrieves the right document and answers wrongly

A retrieval-augmented system answers questions over 60,000 internal documents. Retrieval is measured at recall@5 of 0.94 — the correct document is in the top five results 94% of the time. Answer accuracy, graded by humans, is 0.61.

When answers are wrong, inspection usually shows the correct document WAS in the context. The model produced a fluent answer contradicting it, or blended it with a plausible but absent detail.

The generation prompt places the five retrieved chunks first, then the question, then an instruction to answer only from the provided context.

Retrieval is nearly solved and the answer is still wrong 39% of the time. Enumerate the distinct failure classes that live in the gap between retrieval and generation, and for each the measurement that isolates it.