Retrieval-augmented generation is often adopted specifically to reduce hallucination: give the model the right documents and it will answer from them rather than inventing. It helps, and it does not eliminate the problem. A RAG system can retrieve the wrong material and answer confidently from it, retrieve the right material and ignore it, or blend retrieved facts with invented ones in a way that reads as authoritative. Knowing which of these is happening requires evaluating the system properly, and most teams evaluate only the final answer, which cannot tell them.
For the architecture, ourretrieval-augmented generation overview covers how RAG works. This guide covers proving it works.
Two Failure Points, Not One
The single most important principle in RAG evaluation: the system has two stages that can each fail independently, and evaluating the final answer alone tells you something failed without telling you which.
Retrieval can fail by returning irrelevant material, missing the relevant material entirely, returning outdated versions, or returning material the user should not have access to.
Generation can fail even with perfect retrieval by ignoring the retrieved content, misreading it, over-generalising from it, or adding unsupported claims alongside supported ones.
A team that only measures answer quality will spend weeks tuning prompts to fix what is actually a retrieval problem, or rebuilding retrieval to fix what is actually a generation problem.
Evaluating Retrieval
Retrieval evaluation asks whether the right material reached the model. The practical measures are whether the relevant material appeared at all, whether it appeared high enough in the results to be included given your context budget, and how much irrelevant material came with it, since noise competes for attention.
Building this requires a set of realistic questions with the genuinely relevant sources identified by someone who knows the corpus. That identification is human work, and it is the foundation everything else rests on. Without it, retrieval quality is unmeasurable and teams fall back to impressions.
Evaluating Groundedness
Groundedness asks a narrow, answerable question: is every claim in the answer supported by the retrieved material? This is the core defense against hallucination in a RAG system, and it is assessed claim by claim rather than as an overall impression.
A grounded answer may still be unhelpful, and an ungrounded answer may still be correct because the model happened to know the fact. Both matter, which is why groundedness and correctness are scored separately. An answer that is correct but ungrounded is a warning sign: the system is not actually using its sources, and it will be wrong the moment its general knowledge is wrong or out of date.
The Dimensions Worth Scoring Separately
Groundedness: every claim traceable to retrieved material.
Answer relevance: does it address what was actually asked?
Completeness: does it cover what the sources support, or stop short?
Correctness: is it factually right, judged against ground truth where available?
Appropriate refusal: when the material does not support an answer, does the system say so rather than improvising? This is one of the most valuable behaviors and one of the least tested.
Citation accuracy: where the system cites sources, do the citations actually support the claims? Systems frequently cite plausibly and inaccurately.
Reporting these separately is what makes evaluation actionable. A single blended score tells you the system is mediocre without telling you what to fix.
Automated and Human Evaluation Together
Automated approaches, including using a strong model to judge groundedness, scale well and are appropriate for regression tracking across many cases. They also inherit the judge model's weaknesses, and they are least reliable on exactly the subtle cases that matter: a claim that is almost supported, a citation that is nearly right, an inference the source implies but does not state.
Human review anchors the automated scores. The practical pattern is automated evaluation across a broad set for continuous tracking, with human review on a sample to verify the automated judgments are tracking reality, plus full human review on the hardest categories. The measurement discipline is the same as any structured review programme, with clear rubrics and measured agreement between reviewers, as covered in ourannotation quality guide.
Building the Evaluation Set
A useful RAG evaluation set covers more than typical questions. It needs questions the corpus answers well, questions it answers partially, and questions it cannot answer at all, since refusal behavior is only testable with the last category. It needs questions where two sources conflict, questions where the answer requires combining several sources, and questions where the obvious-seeming source is outdated. Real user questions belong in it, especially the ones that went wrong.
When Evaluation Points Back at the Data
Often the fix is not in the model or the prompt. Poor retrieval frequently traces to knowledge base preparation: documents chunked badly, missing metadata, duplicates, or stale content that should have been removed. When retrieval evaluation shows the right material never surfaces, the work is usually upstream in the data rather than downstream in the model.
Continuous Evaluation
RAG systems drift for reasons unique to them. The corpus changes as documents are added and updated, models change underneath, and user questions shift over time. Continuous evaluation against a stable set, plus sampling of real production answers, is how teams notice before users do. OurAI quality assurance function covers production monitoring.
Common Questions From US Teams
How do I evaluate RAG answer quality?
Evaluate retrieval and generation separately, then score groundedness, relevance, completeness, correctness, appropriate refusal, and citation accuracy as distinct dimensions rather than one blended score.
Does RAG eliminate hallucination?
No, it reduces it. A RAG system can retrieve wrong material and answer from it, ignore correct material, or blend retrieved facts with invented ones. Measuring groundedness is what reveals which is happening.
What is groundedness?
Whether every claim in an answer is supported by the retrieved material, assessed claim by claim. It is the core defense against hallucination in a retrieval system.
Why score groundedness and correctness separately?
Because an answer can be correct but ungrounded, meaning the system answered from general knowledge rather than its sources. That is a warning sign, since it will fail when its general knowledge is wrong or outdated.
How do I test whether the system refuses appropriately?
Include questions the corpus genuinely cannot answer. Appropriate refusal is only testable with those cases, and it is one of the most valuable and least tested behaviors.
Can I evaluate RAG automatically?
Partly, and it is appropriate for regression tracking at scale. Automated judges are least reliable on subtle cases, so human review on a sample is needed to verify the automated scores track reality.
What if evaluation shows retrieval is failing?
The fix is usually upstream in the data: chunking, metadata, duplicates, or stale content. When the right material never surfaces, tuning the model rarely helps.
How often should a RAG system be evaluated?
Continuously. The corpus changes, models change underneath, and user questions shift, so evaluation against a stable set plus production sampling is the only way to catch drift early.
Working With Prudent Partners
Prudent Partners Private Limited evaluates RAG systems for US teams across both stages: retrieval quality against human-identified relevant sources, and generation quality scored for groundedness, refusal behavior, and citation accuracy, with measured reviewer agreement. Where evaluation points upstream, we also prepare the knowledge base itself. See ourretrieval-augmented generation overview andAI quality assurance function.
The first conversation is a 30-minute scoping call about your RAG system, your corpus, and where answers currently go wrong. No commitment to go further.