Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models

TL;DR

This work addresses intra-memory knowledge conflict in language models caused by conflicting pre-training information. It introduces a mechanistic interpretability framework that combines logit lens, activation patching, and cross-model patching (CMAP) to localize where conflicts are encoded and to causally intervene during inference. Using the SynWikiBio dataset, the authors show that late-layer attention components predominantly encode conflicts and that CMAP can steer outputs toward correct facts with varying effectiveness across models. The approach offers a principled path toward understanding and controlling conflicting parametric knowledge, with implications for robust information tracing and potential knowledge-editing applications in LMs.

Abstract

In language models (LMs), intra-memory knowledge conflict largely arises when inconsistent information about the same event is encoded within the model's parametric knowledge. While prior work has primarily focused on resolving conflicts between a model's internal knowledge and external resources through approaches such as fine-tuning or knowledge editing, the problem of localizing conflicts that originate during pre-training within the model's internal representations remain unexplored. In this work, we design a framework based on mechanistic interpretability methods to identify where and how conflicting knowledge from the pre-training data is encoded within LMs. Our findings contribute to a growing body of evidence that specific internal components of a language model are responsible for encoding conflicting knowledge from pre-training, and we demonstrate how mechanistic interpretability methods can be leveraged to causally intervene in and control conflicting knowledge at inference time.

Figures (10)

• Figure 1: A high-level overview (with example) of our proposed framework.
• Figure 2: Overview of the three-stage pipeline for probing intra-memory knowledge conflict.
• Figure 3: Component-wise Probability Contributions for Attention (Top) and MLP components (Bottom).
• Figure 4: Magnitude of probability change of $t_1$ and $t_2$ at layer 43 after patching with $t_s = t_1$. We observe probability changes of $t_1$ on the left and $t_2$ on the right plots. When $\ell'_{mix}$ is patched with $t_1$, the normal expectation is that $\Delta^{c,l,\uparrow}_{t_1}$ is postive while $\Delta^{c,l,\uparrow}_{t_2}$ is negative.
• Figure 5: Top 6 highest impact attention components per-token across the layers, with effect size (layer $l$, token $t$) as the average probability contribution of the attention component in layer $l$ to token $t$.