ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall

TL;DR

The paper tackles the problem of updating factual knowledge in large language models, particularly for multi-hop factual recall where intermediate implicit subjects drive the reasoning chain. It introduces AcE, a neuron-level, attribution-guided editing framework that identifies and edits query–value pathways to propagate updates through multi-hop reasoning. By uncovering that implicit subjects act as query neurons and that value neurons encode semantically interpretable knowledge, AcE achieves substantial improvements over state-of-the-art baselines (e.g., PMET) on GPT-J and Qwen3-8B, and demonstrates that both query and deeper value layers are essential for robust edits. The findings offer a principled pathway toward interpretable and reliable knowledge editing in LLMs with practical implications for maintaining up-to-date and consistent reasoning over cascading knowledge updates.

Abstract

Large Language Models (LLMs) require efficient knowledge editing (KE) to update factual information, yet existing methods exhibit significant performance decay in multi-hop factual recall. This failure is particularly acute when edits involve intermediate implicit subjects within reasoning chains. Through causal analysis, we reveal that this limitation stems from an oversight of how chained knowledge is dynamically represented and utilized at the neuron level. We discover that during multi hop reasoning, implicit subjects function as query neurons, which sequentially activate corresponding value neurons across transformer layers to accumulate information toward the final answer, a dynamic prior KE work has overlooked. Guided by this insight, we propose ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall, a framework that leverages neuron-level attribution to identify and edit these critical query-value (Q-V) pathways. ACE provides a mechanistically grounded solution for multi-hop KE, empirically outperforming state-of-the-art methods by 9.44% on GPT-J and 37.46% on Qwen3-8B. Our analysis further reveals more fine-grained activation patterns in Qwen3 and demonstrates that the semantic interpretability of value neurons is orchestrated by query-driven accumulation. These findings establish a new pathway for advancing KE capabilities based on the principled understanding of internal reasoning mechanisms.

Figures (6)

• Figure 1: Illustration of a multi-hop factual recall. A multi-hop query requires traversing multiple facts. The diagram shows the original knowledge path (e.g., Mark Trumbo$\rightarrow$Basketball$\rightarrow$USA) and how a knowledge edit (green arrow) can target an intermediate fact, which then requires the model to follow a potentially new chain (Mark Trumbo$\rightarrow$Football$\rightarrow$Italy). The intermediate entity "Football" serves as the implicit subject.
• Figure 2: The Impact of Causal Intervention with semantic-related requests upon LLMs of most important layer, including Nationality, Continent, Capital and Language requests.
• Figure 3: Query layers' log increase and value neurons count by layers in GPT-J. Layer log increase is the importance score calculated by logarithmic difference in Eq. \ref{['importance_score_v']}.
• Figure 4: AcE edits Q-V neurons via attribution: (a) The existing locate-then-edit KE method updates new fact using a single-hop prompt; (b) For multi-hop factual recall tasks, traditional locate-then-edit failed to correct edit the knowledge on query layers, overlooking value neurons; (c) Our AcE identifies critical query layers which activates the value neurons most to edit the knowledge.
• Figure 5: The layer-level log increase through all layer upon one case on Qwen3-8B.