Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STABLE: Gated Continual Learning for Large Language Models

William Hoy, Nurcin Celik

TL;DR

This work targets continual adaptation of large language models without full retraining, addressing catastrophic forgetting through a gated self-editing framework called STABLE. By injecting new knowledge via LoRA adapters and applying a gating budget based on Exact Match drop, Bits increase, or KL divergence, the approach either scales or rejects candidate edits to bound forgetting. Empirical results on Qwen-2.5-7B show EM-based gating yields the strongest cumulative improvement, while gating always reduces distributional drift comparably across strategies, underscoring the impact of gate design on adaptation dynamics. The method integrates SEAL’s self-edit generation with explicit acceptance control, offering a principled path toward reliable, long-lived LLMs that can incrementally evolve while preserving prior knowledge.

Abstract

Large language models (LLMs) increasingly require mechanisms for continual adaptation without full retraining. However, sequential updates can lead to catastrophic forgetting, where new edits degrade previously acquired knowledge. This work presents STABLE, a gated continual self editing framework that constrains forgetting during sequential updates using parameter efficient fine tuning via Low Rank Adaptation (LoRA; see arXiv:2106.09685). Each candidate edit is evaluated against a stability budget using one of three metrics: (i) Exact Match (EM) drop, capturing factual accuracy loss; (ii) bits increase, reflecting reduced model confidence; and (iii) KL divergence, quantifying distributional drift between the base and adapted models. If a threshold is exceeded, the LoRA update is rescaled through a clipping procedure or rejected. Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability. EM based gating achieved the highest cumulative performance in short continual learning sequences. Our results show that different gating strategies can achieve comparable distribution shift (measured by KL divergence) while producing different accuracy outcomes, highlighting the importance of gating design in continual adaptation. This approach offers a principled method for continual model editing, enabling LLMs to integrate new knowledge while maintaining reliability. Code: https://github.com/Bhoy1/STABLE

STABLE: Gated Continual Learning for Large Language Models

TL;DR

This work targets continual adaptation of large language models without full retraining, addressing catastrophic forgetting through a gated self-editing framework called STABLE. By injecting new knowledge via LoRA adapters and applying a gating budget based on Exact Match drop, Bits increase, or KL divergence, the approach either scales or rejects candidate edits to bound forgetting. Empirical results on Qwen-2.5-7B show EM-based gating yields the strongest cumulative improvement, while gating always reduces distributional drift comparably across strategies, underscoring the impact of gate design on adaptation dynamics. The method integrates SEAL’s self-edit generation with explicit acceptance control, offering a principled path toward reliable, long-lived LLMs that can incrementally evolve while preserving prior knowledge.

Abstract

Large language models (LLMs) increasingly require mechanisms for continual adaptation without full retraining. However, sequential updates can lead to catastrophic forgetting, where new edits degrade previously acquired knowledge. This work presents STABLE, a gated continual self editing framework that constrains forgetting during sequential updates using parameter efficient fine tuning via Low Rank Adaptation (LoRA; see arXiv:2106.09685). Each candidate edit is evaluated against a stability budget using one of three metrics: (i) Exact Match (EM) drop, capturing factual accuracy loss; (ii) bits increase, reflecting reduced model confidence; and (iii) KL divergence, quantifying distributional drift between the base and adapted models. If a threshold is exceeded, the LoRA update is rescaled through a clipping procedure or rejected. Experiments on the Qwen-2.5-7B model show that gating effectively mitigates forgetting while preserving adaptability. EM based gating achieved the highest cumulative performance in short continual learning sequences. Our results show that different gating strategies can achieve comparable distribution shift (measured by KL divergence) while producing different accuracy outcomes, highlighting the importance of gating design in continual adaptation. This approach offers a principled method for continual model editing, enabling LLMs to integrate new knowledge while maintaining reliability. Code: https://github.com/Bhoy1/STABLE
Paper Structure (27 sections, 11 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 27 sections, 11 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Catastrophic Forgetting
  4. PEFT
  5. Self Adaptive Language Models
  6. Constrained Optimization in Model Updates
  7. Methodology
  8. Gated Continual Self Editing
  9. Formulation
  10. Exact Match (EM) Drop.
  11. Bits Increase.
  12. KL Divergence.
  13. Gate Logic
  14. Results and Analysis
  15. Discussion
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Comparison of unconstrained LoRA merging (top) versus gated merging (bottom). The gate ensures edits are validated for forgetting before merges are accepted, providing a critical safeguard against performance degradation.
  • Figure 2: Performance and drift comparison across gating methods. EM based gating achieves the highest cumulative performance gain (+0.397).
  • Figure 3: Cumulative performance improvement across sequential updates for the three gating methods. EM based gating shows best results per step and cumulative.
  • Figure 4: Cumulative KL divergence across sequential updates for the three gating methods. Dashed horizontal lines indicate the overall KL from baseline to final adapted model, which differs from the sum of per step KL values.