Beyond Retention: Orchestrating Structural Safety and Plasticity in Continual Learning for LLMs

TL;DR

The paper tackles continual learning for large language models by revealing a fundamental split in how Experience Replay affects knowledge: it strengthens robust, unstructured capabilities but severely harms fragile, structured tasks like code generation. It answers this with Orthogonal Subspace Wake-up (OSW), a two-phase, LoRA-based method that identifies critical historical subspaces via a brief anchor wake-up and enforces updates in the orthogonal complement to guarantee structural safety. Empirical results on a four-task sequence show OSW uniquely preserves code generation while maintaining high plasticity for a new task, outperforming ER and other baselines on the safety-plasticity trade-off. The work emphasizes evaluating structural safety alongside average retention in CL for LLMs and demonstrates a scalable approach to protect critical knowledge structures in foundation models.

Abstract

Continual learning in Large Language Models (LLMs) faces the critical challenge of balancing stability (retaining old knowledge) and plasticity (learning new tasks). While Experience Replay (ER) is a standard countermeasure against catastrophic forgetting, its impact across diverse capabilities remains underexplored. In this work, we uncover a critical dichotomy in ER's behavior: while it induces positive backward transfer on robust, unstructured tasks (e.g., boosting performance on previous NLP classification tasks through repeated rehearsal), it causes severe negative transfer on fragile, structured domains like code generation (e.g., a significant relative drop in coding accuracy). This reveals that ER trades structural integrity for broad consolidation. To address this dilemma, we propose \textbf{Orthogonal Subspace Wake-up (OSW)}. OSW identifies essential parameter subspaces of previous tasks via a brief "wake-up" phase and enforces orthogonal updates for new tasks, providing a mathematically grounded "safety guarantee" for established knowledge structures. Empirical results across a diverse four-task sequence demonstrate that OSW uniquely succeeds in preserving fragile coding abilities where Replay fails, while simultaneously maintaining high plasticity for novel tasks. Our findings emphasize the necessity of evaluating structural safety alongside average retention in LLM continual learning.

Figures (2)

• Figure 1: Geometric Illustration of Orthogonal Subspace Wake-up (OSW). This schematic demonstrates how OSW achieves interference-free knowledge recovery. To prevent negative transfer to the new task (whose gradient direction is shown in red), the update vector derived from episodic memory is projected onto the Null Space. The resulting safe update ($\Delta \theta_{wake}$, shown in green) ensures that the retrieval of historical knowledge is mathematically orthogonal to the current optimization landscape, preserving the model's general capabilities.
• Figure 2: Performance dynamics across the four-task continual learning sequence (R1 to R4). These plots visually summarize the critical trade-off between consolidation and structural safety. While Replay (blue squares, dashed line) excels at consolidating robust NLP tasks (T1 & T2) through continual rehearsal, it causes catastrophic degradation on the fragile code task (T3, shaded in red) by Round 4. In stark contrast, our proposed OSW method (red circles, solid line) successfully provides a structural safety guarantee, keeping the fragile code task stable. Simultaneously, OSW maintains high plasticity for the new task (T4, shaded in green), matching the baseline performance (note: OSW and Seq points overlap at 73.0 in T4).