Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation

TL;DR

RALMs combine external retrieval with LLMs but risk unfaithful outputs. SynCheck provides synchronous, multi-signal faithfulness monitoring during decoding, and FOD uses those signals to guide decoding for improved faithfulness and informativeness. Empirical results show SynCheck achieves AUROC around $0.85$ across multiple tasks and models, while FOD delivers substantial gains over abstention, reranking, and CAD, with robust cross-task and cross-model transfer. This approach offers real-time, interpretable, and scalable improvements to reliability in knowledge-intensive generation tasks.

Abstract

Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks. However, there are significant trustworthiness concerns as RALMs are prone to generating unfaithful outputs, including baseless information or contradictions with the retrieved context. This paper proposes SynCheck, a lightweight monitor that leverages fine-grained decoding dynamics including sequence likelihood, uncertainty quantification, context influence, and semantic alignment to synchronously detect unfaithful sentences. By integrating efficiently measurable and complementary signals, SynCheck enables accurate and immediate feedback and intervention, achieving 0.85 AUROC in detecting faithfulness errors across six long-form retrieval-augmented generation tasks, improving prior best method by 4%. Leveraging SynCheck, we further introduce FOD, a faithfulness-oriented decoding algorithm guided by beam search for long-form retrieval-augmented generation. Empirical results demonstrate that FOD outperforms traditional strategies such as abstention, reranking, or contrastive decoding significantly in terms of faithfulness, achieving over 10% improvement across six datasets.

Figures (8)

• Figure 1: (a) An illustration of SynCheck, a fine-grained faithfulness checker for RALMs. SynCheck aggregates real-time decoding dynamics to accurately judge whether a sentence is trustworthy or not. (b) Leveraging SynCheck, we design a two-staged faithfulness-oriented decoding pipeline consisting of backtracking and beam search. In our algorithm, threshold prune takes place before beam size prune, and we use beam size 2 in the example.
• Figure 2: Faithfulness score on Llama 2 7B chat with output truncated to the first $L$ sentences.
• Figure 3: Feature ablation study with SynCheckMLP used as the detection model. We report the AUROC averaged across three runs with different random seeds.
• Figure 4: Performance of SynCheckMLP on different train-test task pairs using Llama 2 7B Chat. We report the AUROC averaged across three runs.
• Figure 5: A list of the well-known entities included in the F-100 and F-100-anti biography generation dataset.