Knowledge Conflicts for LLMs: A Survey

TL;DR

This survey dissects knowledge conflicts in LLMs by separating context-derived information from parametric memory into three conflict types: context-memory, inter-context, and intra-memory. It surveys causes, behavioral analyses, and a wide range of mitigation strategies, including knowledge editing, retrieval-augmented generation, prompting, and discriminative approaches, while highlighting gaps between artificial benchmarks and real-world deployment. The work emphasizes that model behavior hinges on the specific conflict type and that existing solutions often target a subset of scenarios, calling for more nuanced, cross-cutting methods. Overall, the paper provides a taxonomy and a roadmap for building more robust, trustworthy LLMs in dynamic information environments, with attention to downstream impact and real-world applicability.

Abstract

This survey provides an in-depth analysis of knowledge conflicts for large language models (LLMs), highlighting the complex challenges they encounter when blending contextual and parametric knowledge. Our focus is on three categories of knowledge conflicts: context-memory, inter-context, and intra-memory conflict. These conflicts can significantly impact the trustworthiness and performance of LLMs, especially in real-world applications where noise and misinformation are common. By categorizing these conflicts, exploring the causes, examining the behaviors of LLMs under such conflicts, and reviewing available solutions, this survey aims to shed light on strategies for improving the robustness of LLMs, thereby serving as a valuable resource for advancing research in this evolving area.

Figures (4)

• Figure 1: An LLM may encounter three distinct types of knowledge conflicts, stemming from knowledge sources—either contextual (I. Context, yellow chatboxes) or inherent to the LLM's parameters (II. Memory, blue chatboxes). When confronted with a user's question (purple chatbox) entailing knowledge of complex conflicts, the LLM is required to resolve these discrepancies to deliver accurate responses.
• Figure 2: We view knowledge conflict not only as a standalone phenomenon but also as a nexus that connects various causal triggers (causes) with the behaviors of LLMs. While existing literature mainly focuses on II. Analysis, our survey involves systematically observing these conflicts, offering insights into their emergence and impact on LLMs' behavior, along with the desirable behaviors and related solutions.
• Figure 3: Taxonomy of knowledge conflicts. We mainly list works in the era of LLMs. denotes pre-hoc solution and denotes post-hoc solution.
• Figure :