Influence Guided Context Selection for Effective Retrieval-Augmented Generation

TL;DR

The paper tackles RAG hallucinations caused by noisy contexts by introducing Contextual Influence value (CI value), defined as φ_i(v) = v(C) − v(C ∖ {c_i}), which jointly captures query-, list-, and generator-aware signals. A hierarchical CI Surrogate Model (CSM) predicts CI values at inference time, with two training paradigms: supervised learning using oracle CI values and end-to-end training that leverages generator feedback via a differentiable masking mechanism. Experiments across 8 knowledge-intensive tasks and two LLM backbones show that CI-based context filtering substantially improves generation quality, achieving strong correlation to oracle CI (ρ > 0.75) and about a 15% average gain over baselines, while reducing latency. The work offers a scalable, hyperparameter-free approach to context selection in RAG, with potential for broad impact on grounding and reliability in knowledge-intensive NLP applications.

Abstract

Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at https://github.com/SJTU-DMTai/RAG-CSM.

Figures (12)

• Figure 1: An example demonstrating different context quality metrics in practice. $c_2$ and $c_4$ are golden contexts containing crucial information about Trump's second presidency and his vice president Vance. Query-aware metrics favor $c_1$ and $c_3$ due to their mentions of "Trump" and "vice president". List-aware metrics score $c_2$ and $c_4$ higher by considering context relationships, but still favor $c_1$.Generator-aware metrics assign low scores to $c_1$ as it's redundant with LLM knowledge. CI value, by integrating all three dimensions, correctly identifies $c_2$ and $c_4$ as the most informative contexts.
• Figure 2: Overview of our proposed CSM: (a) CSM model architecture, (b) supervised training paradigm and (c) end-to-end training paradigm.
• Figure 3: Context selection by adding high-quality contexts.
• Figure 4: Context selection by adding poor-quality contexts.
• Figure 6: The Spearman correlation ($\rho$) of CSM's predictions with the oracle CI values.

Theorems & Definitions (1)

• Definition 1: Contextual Influence