Real Time Detection and Quantitative Analysis of Spurious Forgetting in Continual Learning

TL;DR

The shallow versus deep alignment framework is introduced, providing the first quantitative characterization of alignment depth, and explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective.

Abstract

Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss. However, this work only qualitatively describes alignment, relies on post-hoc analysis, and lacks automatic distinction mechanisms. We introduce the shallow versus deep alignment framework, providing the first quantitative characterization of alignment depth. We identify that current task alignment approaches suffer from shallow alignment - maintained only over the first few output tokens (approximately 3-5) - making models vulnerable to forgetting. This explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective. We propose a comprehensive framework addressing all gaps: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment. Extensive experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy and show that promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.

Figures (6)

• Figure 1: Experimental Group 1: Baseline Control workflow. Tasks are trained sequentially using standard continual learning (3 epochs per task) without any mitigation strategies. Performance is evaluated on all tasks after each new task is learned, showing natural catastrophic forgetting behavior.
• Figure 2: Experimental Group 2: Spurious Forgetting Induced workflow. Freezing bottom 30% layers disrupts shallow alignment (alignment depth $D \leq 3$) while preserving deep representations (high representation similarity $> 0.85$). This induces spurious forgetting characterized by high reversibility ($R > 0.6$) and fast recovery time.
• Figure 3: Experimental Group 3: True Forgetting Induced workflow. High-intensity training (10 epochs) with minimal old task data causes fundamental changes in representation space, leading to true knowledge loss. This is characterized by low reversibility ($R \leq 0.6$), low representation similarity ($< 0.67$), and slow recovery time (115-168 seconds).
• Figure 4: Experimental Group 4: Mixed Forgetting workflow. Different tasks experience different forgetting types: Task 1 undergoes layer freezing (spurious forgetting), Task 2 undergoes high-intensity training (true forgetting), and Task 3 uses standard training. The framework must correctly identify and distinguish these different forgetting types within the same experimental run.
• Figure 5: Experimental Group 5: Deep Alignment Training workflow. Three specialized training strategies work together: (1) Token-Position Weighted Loss assigns higher weights to later token positions; (2) Multi-Position Alignment Regularization ensures alignment across multiple positions; (3) Sequential Alignment Training validates alignment depth during training. These strategies promote alignment depth from $D \leq 3$ to $D > 12$, significantly improving robustness.