Probing Latent Knowledge Conflict for Faithful Retrieval-Augmented Generation

TL;DR

This work tackles contextual faithfulness in retrieval-augmented generation by interrogating how LLMs internally fuse retrieved evidence with parametric memory under knowledge conflicts. It reveals that knowledge integration is hierarchical and that sentence-level representations encode a latent conflict signal, while irrelevant context can be amplified. Based on these insights, it introduces CLEAR, a three-part framework consisting of fine-grained knowledge pruning, hidden-state conflict probing, and conflict-aware fine-tuning with an attention guidance loss. Across three benchmarks and multiple backbones, CLEAR achieves state-of-the-art results, improving both factual accuracy and faithfulness and demonstrating robust generalization to diverse conflict scenarios.

Abstract

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm to enhance the factuality of Large Language Models (LLMs). However, existing RAG systems often suffer from an unfaithfulness issue, where the model's response contradicts evidence from the retrieved context. Existing approaches to improving contextual faithfulness largely rely on external interventions, such as prompt engineering, decoding constraints, or reward-based fine-tuning. These works treat the LLM as a black box and overlook a crucial question: how does the LLM internally integrate retrieved evidence with its parametric memory, particularly under knowledge conflicts? To address this gap, we conduct a probing-based analysis of hidden-state representations in LLMs and observe three findings: knowledge integration occurs hierarchically, conflicts manifest as latent signals at the sentence level, and irrelevant context is often amplified when aligned with parametric knowledge. Building on these findings, we propose CLEAR (Conflict-Localized and Enhanced Attention for RAG), a framework that (i) decomposes context into fine-grained sentence-level knowledge, (ii) employs hidden-state probing to localize conflicting knowledge, and (iii) introduces conflict-aware fine-tuning to guide the model to accurately integrate retrieved evidence. Extensive experiments across three benchmarks demonstrate that CLEAR substantially improves both accuracy and contextual faithfulness, consistently outperforming strong baselines under diverse conflict conditions. The related resources are available at https://github.com/LinfengGao/CLEAR.

Figures (5)

• Figure 1: Preliminary analysis of contextual unfaithfulness in RAG reveals that all models degrade when (i) exposed to irrelevant knowledge or (ii) confronted with conflicting knowledge.
• Figure 2: t-SNE visualization of hidden-state patterns between aligned and conflicting knowledge. There is a clear distinction in the distribution of hidden states between aligned and conflicting knowledge. This observation provides empirical support for detecting knowledge conflicts based on hidden state representations.
• Figure 3: The overview of our proposed framework CLEAR, which consists of three main components: (i) Fine-Grained Knowledge Pruning, which extracts knowledge from the context and filters out irrelevant items; (ii) Hidden-State Probing for Conflict Detection, which trains a probing model for detecting knowledge conflict by observing hidden state; (iii) Conflict-Aware Fine-Tuning, which regularizes the LLM's attention distribution on conflict content by fine-tuning through an auxiliary attention loss.
• Figure 4: Impact of $\alpha$ on accuracy (blue) and attention weight on conflicting knowledge (red) across different models. Results show that increasing $\alpha$ consistently increases the attention weight assigned to conflicting knowledge. Model performance peaks at smaller $\alpha$ values (0.1 to 0.3) and then declines, indicating that excessive focus on conflicting knowledge can negatively affect performance.
• Figure 5: Context decomposition prompt used in the Fine-Grained Knowledge Pruning module.