Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncovering Context Reliance in Unstructured Knowledge Editing

Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren

TL;DR

The paper identifies Context Reliance as a fundamental failure mode of next-token prediction (NTP) when editing unstructured knowledge in LLMs, where learned facts become tethered to preceding text and fail to recall when that context is absent. It provides empirical evidence and a gradient-based theoretical explanation for why this context dependence arises, then proposes COIN, a simple framework combining Context Alignment Loss $\mathcal{L}_{\text{align}}$ and Knowledge Consistency Loss $\mathcal{L}_{\text{cons}}$ to decouple knowledge from contextual patterns. Empirical results on Llama3-8B and Qwen2.5-7B across AKEW, UnKEBench, and MQuAKE show COIN substantially mitigates Context Reliance (about 45.2% reduction) and improves unstructured editing success by 25.6%, with strong generalization to structured, multi-hop tasks. This work highlights a path toward robust, context-robust unstructured knowledge editing by rethinking training/edit objectives to internalize knowledge independently of its textual surroundings.

Abstract

Editing Large language models (LLMs) with real-world, unstructured knowledge is essential for correcting and updating their internal parametric knowledge. In this work, we revisit the fundamental next-token prediction (NTP) as a candidate paradigm for unstructured editing. We identify Context Reliance as a critical failure mode of NTP-based approaches, where knowledge acquired from edited text becomes highly dependent on its preceding context, leading to recall failures when that context is absent during inference. This hypothesis is supported by our empirical validation that prepending context during inference recovers knowledge recall. We further theoretically demonstrate that Context Reliance is an inherent consequence of gradient-based optimization, which tends to bind acquired knowledge to a specific aggregated contextual representation. To address this, we propose a simple yet effective COntext-INdependent editing framework (COIN), encouraging model to focus on knowledge within local scope rather than memorizing contextual patterns. Evaluations show that COIN reduces Context Reliance by 45.2% and outperforms strong baselines by 23.6% in editing success rate, highlighting the vital role of mitigating Context Reliance for robust editing.

Uncovering Context Reliance in Unstructured Knowledge Editing

TL;DR

The paper identifies Context Reliance as a fundamental failure mode of next-token prediction (NTP) when editing unstructured knowledge in LLMs, where learned facts become tethered to preceding text and fail to recall when that context is absent. It provides empirical evidence and a gradient-based theoretical explanation for why this context dependence arises, then proposes COIN, a simple framework combining Context Alignment Loss and Knowledge Consistency Loss to decouple knowledge from contextual patterns. Empirical results on Llama3-8B and Qwen2.5-7B across AKEW, UnKEBench, and MQuAKE show COIN substantially mitigates Context Reliance (about 45.2% reduction) and improves unstructured editing success by 25.6%, with strong generalization to structured, multi-hop tasks. This work highlights a path toward robust, context-robust unstructured knowledge editing by rethinking training/edit objectives to internalize knowledge independently of its textual surroundings.

Abstract

Editing Large language models (LLMs) with real-world, unstructured knowledge is essential for correcting and updating their internal parametric knowledge. In this work, we revisit the fundamental next-token prediction (NTP) as a candidate paradigm for unstructured editing. We identify Context Reliance as a critical failure mode of NTP-based approaches, where knowledge acquired from edited text becomes highly dependent on its preceding context, leading to recall failures when that context is absent during inference. This hypothesis is supported by our empirical validation that prepending context during inference recovers knowledge recall. We further theoretically demonstrate that Context Reliance is an inherent consequence of gradient-based optimization, which tends to bind acquired knowledge to a specific aggregated contextual representation. To address this, we propose a simple yet effective COntext-INdependent editing framework (COIN), encouraging model to focus on knowledge within local scope rather than memorizing contextual patterns. Evaluations show that COIN reduces Context Reliance by 45.2% and outperforms strong baselines by 23.6% in editing success rate, highlighting the vital role of mitigating Context Reliance for robust editing.
Paper Structure (39 sections, 2 theorems, 24 equations, 14 figures, 8 tables)

This paper contains 39 sections, 2 theorems, 24 equations, 14 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Uncovering Context Reliance in NTP
  4. Experimental Setup
  5. Empirical Analysis: Context Reliance Impairs Knowledge Recall
  6. Theoretical Analysis: Gradient-Based Optimization Induces Context Reliance
  7. Intuitive Mitigation Techniques
  8. Context-Independent Knowledge Editing
  9. Context Alignment Loss
  10. Knowledge Consistency Loss
  11. Experiments
  12. Baselines
  13. Experiments Results
  14. Conclusion
  15. Limitations & Future Discussion
  16. ...and 24 more sections

Key Result

Theorem 3.2

Under the setting of Assumption a:settings, after one gradient update, the model exhibits the following behavior during inference, where $\operatorname{logit}\in\mathbb{R}^M$ denotes the prediction logits. Success with context. If both contextual tokens $p$ and $q$ are present, then Failure without context. If the contextual token $q$ is absent, then where ${\mathbb{A}}$ denotes the set of token

Figures (14)

  • Figure 1: (a) Unstructured text containing multiple pieces of knowledge. (b) Edited model tends to recall knowledge appearing later in the text less accurately. (c) Knowledge acquired from edited text is overly dependent on its preceding context, and omitting that context during inference leads to failed recall.
  • Figure 2: Performance decline of different NTP-based methods on AKEW dataset as the position of knowledge moves later in the text. The x-axis represents the sequential order of knowledge within the original text, where position '1' corresponds to questions targeting knowledge positioned at the beginning, and position '$>$6' corresponds to knowledge located towards the end. The y-axis represents the corresponding accuracy.
  • Figure 3: Probability of predicting answer with and without preceding context. The results show that appending context increases probability of answer.
  • Figure 4: Performance comparison between with and without knowledge splitting (KS).
  • Figure 5: Performance comparison between with and without paraphrasing (Para.).
  • ...and 9 more figures

Theorems & Definitions (3)

  • Theorem 3.2: Context Reliance Induced by Gradient Descent
  • Theorem 2.1: Context Reliance after one gradient step
  • proof