Better retrieval doesn't guarantee better RAG answers: improving individual components can paradoxically increase confident hallucinations when relevant information isn't in your corpus.
This paper studies retrieval-augmented generation (RAG) systems for answering questions about AI policy documents. The researchers found that improving retrieval quality doesn't always lead to better answers—sometimes better retrieval actually makes the system more confidently wrong when relevant documents are missing.