EtCon: Edit-then-Consolidate for Reliable Knowledge Editing

TL;DR

The paper identifies a critical gap in knowledge editing for lifelong learning, namely overfitting to edited facts and the lack of a consolidation stage that aligns parametric updates with autoregressive generation. It introduces Edit-then-Consolidate (EtCon), a two-stage framework combining Targeted Proximal Supervised Fine-Tuning (TPSFT) for localized edits and Group Relative Policy Optimization (GRPO) for trajectory-level consolidation under real-world signals. Extensive experiments on ZsRE, COUNTERFACT, and QAEdit across Llama-3-8B-Instruct and Qwen-2.5-7B-Instruct show substantial improvements in reliability and generalization, with maintained locality and pre-trained capabilities, validating the necessity of the consolidation stage. The work demonstrates that decoupling editing and consolidation yields practical, robust lifelong knowledge editing in autoregressive LLM settings.

Abstract

Knowledge editing aims to update specific facts in large language models (LLMs) without full retraining. Prior efforts sought to tune the knowledge layers of LLMs, proving effective for making selective edits. However, a significant gap exists between their performance in controlled, teacher-forcing evaluations and their real-world effectiveness in lifelong learning scenarios, which greatly limits their practical applicability. This work's empirical analysis reveals two recurring issues associated with this gap: (1) Most traditional methods lead the edited model to overfit to the new fact, thereby degrading pre-trained capabilities; (2) There is a critical absence of a knowledge consolidation stage, leaving new facts insufficiently integrated into LLMs' inference-time behavior under autoregressive generation, thereby leading to a mismatch between parametric knowledge and actual generation behavior. To this end, we propose Edit-then-Consolidate, a novel knowledge editing paradigm that aims to bridge the gap between theoretical knowledge editing methods and their real-world applicability. Specifically, (1) our framework mitigates overfitting via Targeted Proximal Supervised Fine-Tuning (TPSFT) that localizes the edit via a trust-region objective to limit policy drift; (2) Then, a consolidation stage using Group Relative Policy Optimization (GRPO) aligns the edited knowledge with CoT-based inference policy by optimizing trajectory-level behavior under comprehensive reward signals. Extensive experiments demonstrate our framework consistently improves editing reliability and generalization under real-world evaluations, while better preserving locality and pre-trained capabilities.

Figures (12)

• Figure 1: Illustration of the knowledge editing problem and our Edit-then-Consolidate solution.
• Figure 1: Performance comparison w/ and w/o consolidation under real-world evaluation on ZsRE. (+GRPO) denotes adding our consolidation stage.
• Figure 2: Reward curves comparison
• Figure 3: Overview of the Edit-then-Consolidate (EtCon) Framework. Edit stage: We employ Targeted Proximal Supervised Fine-Tuning (TPSFT) to perform localized edits within the selected FFN layers to inject new knowledge. Consolidate stage: We use Group Relative Policy Optimization (GRPO) with a comprehensive reward function to align the parametric knowledge with CoT-based inference policy.
• Figure 4: The complete prompt used to employ a LLM as a judge for providing binary assessments (correct or incorrect) based on a given question, gold target answer, and predicted answer.