Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing

Xinyu Wang, Sicheng Lyu, Yu Gu, Jerry Huang, Peng Lu, Yufei Cui, Xiao-Wen Chang

TL;DR

This work tackles the challenge of lifelong sequential editing of large language models by introducing RLSEdit, a recursive least-squares editor that treats edits as an online quadratic optimization with two soft constraints. By penalizing deviation from the initial weights and from an anchor mapping, RLSEdit enables efficient per-edit updates via the Woodbury identity, yielding a per-edit cost that scales with the current edit size rather than the history length. Theoretical results provide deviation bounds and an asymptotic characterization, while extensive experiments on Llama3-8B and Qwen2.5-7B show stable scaling to 10K edits, superior edit efficacy, and strong preservation of general language understanding and reasoning across GLUE and code/math benchmarks. The method offers a practical, interpretable balance between adapting to new information and retaining prior capabilities in long-running editing scenarios.

Abstract

Model editing updates a pre-trained LLM with new facts or rules without re-training, while preserving unrelated behavior. In real deployment, edits arrive as long streams, and existing editors often face a plasticity-stability dilemma: locate-then-edit "hard writes" can accumulate interference over time, while null-space-style "hard preservation" preserves only what is explicitly constrained, so past edits can be overwritten and unconstrained behaviors may deviate, degrading general capabilities in the many-edits regime. We propose RLSEdit, a recursive least-squares editor for long sequential editing. RLSEdit formulates editing as an online quadratic optimization with soft constraints, minimizing a cumulative key-value fitting objective with two regularizers that control for both deviation from the pre-trained weights and from a designated anchor mapping. The resulting update admits an efficient online recursion via the Woodbury identity, with per-edit cost independent of history length and scaling only with the current edit size. We further provide deviation bounds and an asymptotic characterization of the adherence-preservation trade-off in the many-edits regime. Experiments on multiple model families demonstrate stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability -- crucially retaining early edits, and preserving general capabilities on GLUE and held-out reasoning/code benchmarks.

Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing

TL;DR

This work tackles the challenge of lifelong sequential editing of large language models by introducing RLSEdit, a recursive least-squares editor that treats edits as an online quadratic optimization with two soft constraints. By penalizing deviation from the initial weights and from an anchor mapping, RLSEdit enables efficient per-edit updates via the Woodbury identity, yielding a per-edit cost that scales with the current edit size rather than the history length. Theoretical results provide deviation bounds and an asymptotic characterization, while extensive experiments on Llama3-8B and Qwen2.5-7B show stable scaling to 10K edits, superior edit efficacy, and strong preservation of general language understanding and reasoning across GLUE and code/math benchmarks. The method offers a practical, interpretable balance between adapting to new information and retaining prior capabilities in long-running editing scenarios.

Abstract

Model editing updates a pre-trained LLM with new facts or rules without re-training, while preserving unrelated behavior. In real deployment, edits arrive as long streams, and existing editors often face a plasticity-stability dilemma: locate-then-edit "hard writes" can accumulate interference over time, while null-space-style "hard preservation" preserves only what is explicitly constrained, so past edits can be overwritten and unconstrained behaviors may deviate, degrading general capabilities in the many-edits regime. We propose RLSEdit, a recursive least-squares editor for long sequential editing. RLSEdit formulates editing as an online quadratic optimization with soft constraints, minimizing a cumulative key-value fitting objective with two regularizers that control for both deviation from the pre-trained weights and from a designated anchor mapping. The resulting update admits an efficient online recursion via the Woodbury identity, with per-edit cost independent of history length and scaling only with the current edit size. We further provide deviation bounds and an asymptotic characterization of the adherence-preservation trade-off in the many-edits regime. Experiments on multiple model families demonstrate stable scaling to 10K edits, outperforming strong baselines in both edit success and holistic stability -- crucially retaining early edits, and preserving general capabilities on GLUE and held-out reasoning/code benchmarks.
Paper Structure (39 sections, 4 theorems, 71 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 39 sections, 4 theorems, 71 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Writing and Preservation Trade-off.
  4. Moving to Longer Edit Streams.
  5. Soft Updates with Preservation.
  6. Beyond Direct Parameter Writes.
  7. Methodology
  8. A recursive least squares editor
  9. Setup
  10. Regularized Least-Squares Equation
  11. Efficient Recursion via Normal Equations
  12. Complexity analysis
  13. RLS-Woodbury Updates.
  14. Comparison to other sequential editors.
  15. Hard versus Soft Constraints
  16. ...and 24 more sections

Key Result

Theorem 4.1

Let ${\bm{W}}_t^*$ be the minimizer of $J_t({\bm{W}})$ and define ${\bm{R}}_t\coloneq{\bm{V}}_t-{\bm{K}}_t{\bm{W}}_{t\!-\!1}^*$. Let $\sigma_{\min}({\bm{K}})$ denotes the smallest singular value of ${\bm{K}}$.

Figures (7)

  • Figure 1: The recursive workflow of our RLS-Woodbury editor. The process alternates between updating the covariance state via the Woodbury identity (Phase 1) and updating weights (Phase 2). The highlighted block shows how we reduce complexity from $O\!\left(d_k^3\right)$ to $O\!\left(d_k^2 u_t\right)$ by solving small $u_t\!\times\!u_t$ systems.
  • Figure 2: Evolution of objective terms over $10\mathsf{K}$ edits. We compare RLSEdit against baselines (AlphaEdit, MEMIT) on three metrics: Term 1 ($\|{\bm{K}}_t {\bm{W}}-{\bm{V}}_t\|_F^2$) measures the fitting error for the current edit; Term 2 ($\|{\bm{W}}-{\bm{W}}_0\|_F^2$) measures parameter drift from the initial weights; and Term 3 ($\|{\bm{K}}_0 {\bm{W}}-{\bm{V}}_0\|_F^2$) measures the preservation error on the preserved knowledge. The results show that RLSEdit consistently maintains lower values across all three terms, supporting the stability of our soft-constraint formulation.
  • Figure 3: General capability preservation. We evaluate 5 GLUE tasks and additional benchmarks for general knowledge, math reasoning and coding ability (MMLU, GSM8K, HumanEval, MBPP) at multiple editing checkpoints (Pre-edit, 2k--10k edits). RLSEdit is compared against baselines and consistently better preserves the model’s general capabilities across tasks and edit scales. The x-axis shows the cumulative number of applied edits, and the y-axis reports the corresponding score (F1 or accuracy).
  • Figure 4: Improvements on early edits. After applying 10K sequential edits, we re-evaluate performance on the earliest edited cases (500, 1K, 2K, 4K). Each bar reports the Rewrite or Paraphrase score. RLSEdit consistently achieves the highest scores across all settings.
  • Figure 5: Case study on HumanEval task 0 (AlphaEdit). The prompt above is the original statement of HumanEval/0. AlphaEdit remains correct at 2k edits but begins to fail from 4k edits onward (boundary error), later exhibiting a semantic bug at 8k and degenerating into near-empty/garbled output at 10k.
  • ...and 2 more figures

Theorems & Definitions (9)

  • Remark 3.1
  • Theorem 4.1: Global deviation bounds
  • Proposition 4.2: Asymptotic behavior of the RLS editor
  • proof : Proof of Theorem \ref{['prop:global_deviation']}(i) (parameter deviation)
  • Lemma A.1
  • proof
  • proof : Proof of Theorem \ref{['prop:global_deviation']}(ii) and the adaptive spectral variant
  • Corollary A.2: Hard limits from soft penalties
  • proof