Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimization

TL;DR

This work identifies a salient correlation between LFQA faithfulness and retrieval heads, a set of attention heads responsible for retrieving contextual information and proposes RHIO, a framework designed to teach LLMs to explicitly discriminate between faithful and unfaithful generations.

Abstract

Ensuring contextual faithfulness in retrieval-augmented large language models (LLMs) is crucial for building trustworthy information-seeking systems, particularly in long-form question-answering (LFQA) scenarios. In this work, we identify a salient correlation between LFQA faithfulness and retrieval heads, a set of attention heads responsible for retrieving contextual information. Leveraging this insight, we propose RHIO, a framework designed to teach LLMs to explicitly discriminate between faithful and unfaithful generations. RHIO first augments unfaithful samples that simulate realistic model-intrinsic errors by selectively masking retrieval heads. Then, these samples are incorporated into joint training, enabling the model to distinguish unfaithful outputs from faithful ones conditioned on control tokens. Furthermore, these control tokens are leveraged to self-induce contrastive outputs, amplifying their difference through contrastive decoding. Additionally, to facilitate the evaluation of contextual faithfulness, we also introduce GroundBench, a comprehensive benchmark compiled from five existing LFQA datasets. Extensive experimental results on GroundBench demonstrate that RHIO significantly improves faithfulness, even outperforming GPT-4o.

Figures (11)

• Figure 1: Impact of masking different numbers of masked retrieval heads on model faithfulness.
• Figure 3: An overview of Rhio: (1) unfaithful data augmentation (§\ref{['ssec:negative_augmentation']}), which augments unfaithful output by masking out attention heads responsible for contextual faithfulness; (2) faithfulness-aware tuning (§\ref{['ssec:learning_to_discriminate']}), which teaches LLMs to explicitly discriminate between faithful and unfaithful outputs; (3) self-induced decoding (§\ref{['ssec:unfaithful_induced_decoding']}), which further enhances faithfulness by amplifying the differences between induced contrastive outputs.
• Figure 4: Ablation study on hyperparameter $\alpha$ and the decoding strategy in self-induced decoding.
• Figure 5: Ablation study on different negative sample augmentation strategies.
• Figure 6: Illustration of the prompting used for training and evaluation.