Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Consistency-Aware Parameter-Preserving Knowledge Editing Framework for Multi-Hop Question Answering

Lingwen Deng, Yifei Han, Shijie Li, Yue Du, Bin Li

TL;DR

CAPE-KG tackles inconsistency in KG-based parameter-preserving knowledge editing for multi-hop QA by enforcing knowledge boundary, update, and intent consistency. It introduces a multi-layer KG with a base factual layer and per-case overlays, plus an isolated update mechanism and edit-aware retrieval routing that directs queries to the appropriate layer. Across the MQuAKE-CF-3K and MQuAKE-T benchmarks, CAPE-KG achieves substantial gains in M-Acc and H-Acc while remaining model-agnostic across backbones such as LLaMA-2-7B, Vicuna-7B, and GPT-3.5-turbo-instruct, demonstrating robust, consistent reasoning under edits. The framework offers a practical, parameter-preserving path to reliable knowledge edits in MHQA, with noted latency trade-offs from progressive retrieval and an assumption of case isolation for edits.

Abstract

Parameter-Preserving Knowledge Editing (PPKE) enables updating models with new information without retraining or parameter adjustment. Recent PPKE approaches used knowledge graphs (KG) to extend knowledge editing (KE) capabilities to multi-hop question answering (MHQA). However, these methods often lack consistency, leading to knowledge contamination, unstable updates, and retrieval behaviors that are misaligned with the intended edits. Such inconsistencies undermine the reliability of PPKE in multi-hop reasoning. We present CAPE-KG, Consistency-Aware Parameter-Preserving Editing with Knowledge Graphs, a novel consistency-aware framework for PPKE on MHQA. CAPE-KG ensures KG construction, update, and retrieval are always aligned with the requirements of the MHQA task, maintaining coherent reasoning over both unedited and edited knowledge. Extensive experiments on the MQuAKE benchmark show accuracy improvements in PPKE performance for MHQA, demonstrating the effectiveness of addressing consistency in PPKE.

Consistency-Aware Parameter-Preserving Knowledge Editing Framework for Multi-Hop Question Answering

TL;DR

CAPE-KG tackles inconsistency in KG-based parameter-preserving knowledge editing for multi-hop QA by enforcing knowledge boundary, update, and intent consistency. It introduces a multi-layer KG with a base factual layer and per-case overlays, plus an isolated update mechanism and edit-aware retrieval routing that directs queries to the appropriate layer. Across the MQuAKE-CF-3K and MQuAKE-T benchmarks, CAPE-KG achieves substantial gains in M-Acc and H-Acc while remaining model-agnostic across backbones such as LLaMA-2-7B, Vicuna-7B, and GPT-3.5-turbo-instruct, demonstrating robust, consistent reasoning under edits. The framework offers a practical, parameter-preserving path to reliable knowledge edits in MHQA, with noted latency trade-offs from progressive retrieval and an assumption of case isolation for edits.

Abstract

Parameter-Preserving Knowledge Editing (PPKE) enables updating models with new information without retraining or parameter adjustment. Recent PPKE approaches used knowledge graphs (KG) to extend knowledge editing (KE) capabilities to multi-hop question answering (MHQA). However, these methods often lack consistency, leading to knowledge contamination, unstable updates, and retrieval behaviors that are misaligned with the intended edits. Such inconsistencies undermine the reliability of PPKE in multi-hop reasoning. We present CAPE-KG, Consistency-Aware Parameter-Preserving Editing with Knowledge Graphs, a novel consistency-aware framework for PPKE on MHQA. CAPE-KG ensures KG construction, update, and retrieval are always aligned with the requirements of the MHQA task, maintaining coherent reasoning over both unedited and edited knowledge. Extensive experiments on the MQuAKE benchmark show accuracy improvements in PPKE performance for MHQA, demonstrating the effectiveness of addressing consistency in PPKE.

Paper Structure

This paper contains 33 sections, 1 equation, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Method
  4. Knowledge Graph Construction
  5. Knowledge Update
  6. Knowledge Retrieval
  7. Experiment
  8. Experiment Setting
  9. Results
  10. Ablation Studies
  11. Conclusion
  12. Dataset
  13. Baselines
  14. Parameter-Updating Editing Methods
  15. FT
  16. ...and 18 more sections

Figures (3)

  • Figure 1: An overview of CAPE-KG, consists of three major components: the KG Construction module for building a pre-edit KG, the Knowledge Update module for incorporating edited knowledge, and the Knowledge Retrieval module for generating answers via retrieval.
  • Figure 2: Case Comparison of Update and KG Construction. Unedited knowledge is labeled in purple. Edited entities are in yellow, with dotted lines indicating the edited relation.
  • Figure 3: Case Comparison of Intent Consistency in Retrieval.