FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning

TL;DR

FOREVER tackles catastrophic forgetting in LLM continual learning by redefining replay timing and strength through a model-centric timeline based on parameter update magnitude. It introduces a Forgetting Curve-inspired Replay Scheduler that maps human days to model days using accumulated updates $\tau_t$ and a virtual model day $\tau_{\text{day}}$, triggering replays at thresholds $\mathcal{D}_{\text{model}}$. The intensity-aware Replay Regularization modulates replay strength with an instability ratio $r_t$, using $\mathcal{L}_{\text{replay}} = \mathcal{L}_{\text{task}}^{(old)} + \beta_t \sum_j \lVert \Theta_j - \Theta_j^{*} \rVert_2^2$ and $\beta_t$ scaled by $r_t$. Across three CL benchmarks and multiple backbones ($0.6$B to $13$B), FOREVER consistently reduces forgetting and preserves adaptability, outperforming step-based and many replay-based baselines. This model-dynamics–driven approach offers a principled, scalable strategy for memory replay in large-scale continual learning scenarios.

Abstract

Continual learning (CL) for large language models (LLMs) aims to enable sequential knowledge acquisition without catastrophic forgetting. Memory replay methods are widely used for their practicality and effectiveness, but most rely on fixed, step-based heuristics that often misalign with the model's actual learning progress, since identical training steps can result in varying degrees of parameter change. Motivated by recent findings that LLM forgetting mirrors the Ebbinghaus human forgetting curve, we propose FOREVER (FORgEtting curVe-inspired mEmory Replay), a novel CL framework that aligns replay schedules with a model-centric notion of time. FOREVER defines model time using the magnitude of optimizer updates, allowing forgetting curve-inspired replay intervals to align with the model's internal evolution rather than raw training steps. Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay. Extensive experiments on three CL benchmarks and models ranging from 0.6B to 13B parameters demonstrate that FOREVER consistently mitigates catastrophic forgetting.

Figures (7)

• Figure 1: Aligning human time and model time in FOREVER. FOREVER aligns Ebbinghaus-inspired human replay intervals with a model-centric timeline defined by accumulated parameter update magnitude, enabling replay to be triggered based on the model's actual learning progress.
• Figure 2: Overview of FOREVER. FOREVER decomposes replay into two coupled decisions—when to replay and how to replay—both grounded in model update dynamics. Parameter update magnitudes $\Delta_t$ track model evolution over training steps, whose accumulation defines a model-centric notion of time (virtual "model days"). When to replay (Left): accumulated model time $\tau_t$ measures how far the model has progressed in parameter space and triggers replay when Ebbinghaus-guided time thresholds are reached. How to replay (Right): recent update intensity $\mu_t$, relative to a baseline $\mu_0$, modulates replay regularization strength—stronger under rapid model changes and gentler when updates are stable. By unifying replay timing and replay strength under the same update-dynamics signal, FOREVER enables a coherent and model-centric replay strategy.
• Figure 3: Performance of FOREVER with different backbones on the SuperNI Benchmark.
• Figure 4: Impact of Catastrophic Forgetting in Continual Learning. Performance is reported as a percentage of each task's upper bound. After fine-tuning on the final task, FOREVER shows superior resistance to performance decline on previously learned tasks.
• Figure 5: Visualization of model-centric replay dynamics during training.Left: step-wise parameter update magnitude $\Delta_t$ across training steps. Right: accumulated model-centric time $\tau_t$ with replay trigger points annotated. Under the proposed model-centric time definition, replay is triggered at different training steps for different tasks, reflecting task-dependent learning dynamics rather than fixed step-based schedules.