Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness

TL;DR

It is argued that current factuality enhancement methods can significantly undermine the context-faithfulness of LLMs, and recommended that more research on LLMs' factuality enhancement make efforts to reduce the sacrifice of context-faithfulness.

Abstract

As the modern tools of choice for text understanding and generation, large language models (LLMs) are expected to accurately output answers by leveraging the input context. This requires LLMs to possess both context-faithfulness and factual accuracy. Extensive efforts have been made to enable better outputs from LLMs by mitigating hallucinations through factuality enhancement methods. However, they also pose risks of hindering context-faithfulness, as factuality enhancement can lead LLMs to become overly confident in their parametric knowledge, causing them to overlook the relevant input context. In this work, we argue that current factuality enhancement methods can significantly undermine the context-faithfulness of LLMs. We first revisit the current factuality enhancement methods and evaluate their effectiveness in enhancing factual accuracy. Next, we evaluate their performance on knowledge editing tasks to assess the potential impact on context-faithfulness. The experimental results reveal that while these methods may yield inconsistent improvements in factual accuracy, they also cause a more severe decline in context-faithfulness, with the largest decrease reaching a striking 69.7\%. To explain these declines, we analyze the hidden states and logit distributions for the tokens representing new knowledge and parametric knowledge respectively, highlighting the limitations of current approaches. Our finding highlights the complex trade-offs inherent in enhancing LLMs. Therefore, we recommend that more research on LLMs' factuality enhancement make efforts to reduce the sacrifice of context-faithfulness.

Figures (6)

• Figure 1: Comparison of responses from native LLMs and factuality-enhanced LLMs before and after context injection. The enhanced LLMs, due to overconfidence in their parametric knowledge, struggle to integrate new information from the context, resulting in incorrect answers.
• Figure 2: An illustration of the ICE task to evaluate LLMs' context-faithfulness. It involves multi-hop question answering with corresponding edits, utilizing $k$ contextual demonstrations to guide editing and align output format. The highlighted text represents the expected output from the LLMs, while Thoughts indicates the additional step taken when using Chain of Thought (COT) reasoning.
• Figure 3: Changes (%) in Context-Faithfulness and Factuality of Factuality-Enhanced LLMs.
• Figure 4: Average logits of tokens representing new and parametric knowledge after context injection.
• Figure 5: Statistics of new knowledge distribution for both logits and rank on LLaMA2-chat (left) and LLaMA2-13B-chat (right). Rank is recorded by capturing the position of the new knowledge token within the vocabulary $\mathcal{V}$, based on its logits ranking.