RAG-E: Quantifying Retriever-Generator Alignment and Failure Modes

TL;DR

RAG-E introduces an end-to-end explainability framework for Retrieval-Augmented Generation that jointly analyzes how retrievers and generators use and misuse evidence. It combines Integrated Gradients for retriever explanations with Monte Carlo-stabilized Shapley-style attributions for generators (pmcSHAP) and introduces the WARG metric to quantify how well the generator’s document usage aligns with the retriever’s ranking. Empirical results on two retrievers, two generators, and two datasets reveal pervasive misalignment patterns (e.g., primacy bias in generation) and strong evidence of two failure modes: wasted retrieval and noise distraction. The framework enables principled auditing, with open-source tooling, and offers a foundation for improving RAG reliability in high-stakes domains by diagnosing and potentially guiding retrieval/generation alignment decisions.

Abstract

Retrieval-Augmented Generation (RAG) systems combine dense retrievers and language models to ground LLM outputs in retrieved documents. However, the opacity of how these components interact creates challenges for deployment in high-stakes domains. We present RAG-E, an end-to-end explainability framework that quantifies retriever-generator alignment through mathematically grounded attribution methods. Our approach adapts Integrated Gradients for retriever analysis, introduces PMCSHAP, a Monte Carlo-stabilized Shapley Value approximation, for generator attribution, and introduces the Weighted Attribution-Relevance Gap (WARG) metric to measure how well a generator's document usage aligns with a retriever's ranking. Empirical analysis on TREC CAsT and FoodSafeSum reveals critical misalignments: for 47.4% to 66.7% of queries, generators ignore the retriever's top-ranked documents, while 48.1% to 65.9% rely on documents ranked as less relevant. These failure modes demonstrate that RAG output quality depends not solely on individual component performance but on their interplay, which can be audited via RAG-E.

Figures (16)

• Figure 1: Visual example of the generated explanations. We detect important spans influencing both the retrieval and generation steps. This example was generated using Arctic Embed 2 and Llama 3.1.
• Figure 2: RAG-E overview. Explanations are based on intrinsic IG (- -) for the Ret and extrinsic Shapley for the Gen (- -).
• Figure 3: Baseline creation for IG. We replace the embeddings corresponding to non-special tokens with embeddings corresponding to the model's [unk] token evaluated at the same input position. [XXX] denotes a special token and $\phi_i^\text{[XXX]}$ the corresponding embedding at input position $i$.
• Figure 4: MSE compared to precise SV [plots a) & b)] and variance $\sigma^2$ over 10 repetitions [plots c) & d)] of kSHAP, mcSHAP, and pmcSHAP for $|\mathcal{D}| = 5$ (Right column: results for $N=20$; Left column: results for 200 MC samples).
• Figure 5: Attribution Instability on FSS (left) and TC (right).

Theorems & Definitions (3)

• Definition 2.1: Saliency-Based Local Explanation
• Definition 2.2: Additivity
• Definition 3.1: Weighted Attribution-Relevance Gap