Trust Region Continual Learning as an Implicit Meta-Learner

TL;DR

The paper addresses catastrophic forgetting in continual learning by proposing Trust Region Continual Learning (TRCL), a hybrid approach that couples generative replay with a Fisher-metric trust-region constraint. Under local approximations, TRCL induces a one-step, MAML-like update, revealing an implicit meta-learning objective that promotes rapid re-convergence to past task optima after each transition. Empirical results on low-heterogeneity diffusion tasks (ImageNet-500) and high-heterogeneity diffusion-control tasks (CW10) show TRCL achieves stronger retention and faster early-task recovery than replay, EWC, and continual meta-learning baselines, with notable final performance gains (e.g., AFID 44.5 and forgetting 10.6 on ImageNet-500; CW10 SR 88.3% with forgetting 4.4%). This work bridges continual learning and meta-learning at the optimization level, offering a scalable, efficient framework for large diffusion backbones in both image generation and robotic control domains.

Abstract

Continual learning aims to acquire tasks sequentially without catastrophic forgetting, yet standard strategies face a core tradeoff: regularization-based methods (e.g., EWC) can overconstrain updates when task optima are weakly overlapping, while replay-based methods can retain performance but drift due to imperfect replay. We study a hybrid perspective: \emph{trust region continual learning} that combines generative replay with a Fisher-metric trust region constraint. We show that, under local approximations, the resulting update admits a MAML-style interpretation with a single implicit inner step: replay supplies an old-task gradient signal (query-like), while the Fisher-weighted penalty provides an efficient offline curvature shaping (support-like). This yields an emergent meta-learning property in continual learning: the model becomes an initialization that rapidly \emph{re-converges} to prior task optima after each task transition, without explicitly optimizing a bilevel objective. Empirically, on task-incremental diffusion image generation and continual diffusion-policy control, trust region continual learning achieves the best final performance and retention, and consistently recovers early-task performance faster than EWC, replay, and continual meta-learning baselines.

Figures (2)

• Figure 1: Illustrations of parameter updates when learning a new task $\mathcal{T}_3$ under different continual-learning strategies. Colored ellipses denote low-loss regions for each task. Dark arrows show the current update direction from $\theta$, and gray arrows indicate prior trajectory. (a) EWC regularizes updates toward parameters that perform well on previous tasks; if $\mathcal{T}_3$ has little or no overlap with earlier tasks, this constraint can yield no feasible solution (question mark). (b) Generative replay optimizes on the union of current task data and replayed samples from past tasks ($\tilde{\mathcal{T}}_1,\tilde{\mathcal{T}}_2$), allowing convergence to a different low-loss basin, potentially far from earlier optima in overparameterized models with many equivalent solutions. (c) The hybrid approach combines replay with a trust region constraint (red dashed region), encouraging each update to remain within a neighborhood that preserves low error on previous tasks while adapting to $\mathcal{T}_3$.
• Figure 2: Task 1 performance over the course of continual training on 10 tasks, averaged over random seeds, on two datasets. Gray dashed vertical lines mark task transitions. X-axis: gradient update steps.