Context-aware Decoding Reduces Hallucination in Query-focused Summarization

TL;DR

This work addresses hallucinations in query-focused summarization by examining Context-aware Decoding (CAD), a PMI-based product-of-experts decoding strategy. Through a large-scale reproducibility study across eight language models and multiple datasets, CAD is shown to reduce factual errors while largely preserving lexical similarity, albeit with higher inference FLOPs and slower decoding. The study provides a rigorous analysis of hyperparameter sensitivity, particularly the weighting parameter $\alpha$, and demonstrates practical trade-offs between faithfulness and speed. The results support the use of CAD in retrieval-augmented generation and QFS systems where factual fidelity is critical, and the authors release code to support further research and adoption.

Abstract

Query-focused summarization (QFS) aims to provide a summary of a single document/multi documents that can satisfy the information needs of a given query. It is useful for various real-world applications, such as abstractive snippet generation or more recent retrieval augmented generation (RAG). A prototypical QFS pipeline consists of a retriever (sparse or dense retrieval) and a generator (usually a large language model). However, applying large language models (LLM) potentially leads to hallucinations, especially when the evidence contradicts the prior belief of LLMs. There has been growing interest in developing new decoding methods to improve generation quality and reduce hallucination. In this work, we conduct a large-scale reproducibility study on one recently proposed decoding method\, -- \,Context-aware Decoding (CAD). In addition to replicating CAD's experiments on news summarization datasets, we include experiments on QFS datasets, and conduct more rigorous analysis on computational complexity and hyperparameter sensitivity. Experiments with eight different language models show that performance-wise, CAD improves QFS quality by (1) reducing factuality errors/hallucinations while (2) mostly retaining the match of lexical patterns, measured by ROUGE scores, while also at a cost of increased inference-time FLOPs and reduced decoding speed. The \href{https://github.com/zhichaoxu-shufe/context-aware-decoding-qfs}{code implementation} based on Huggingface Library is made available