Adversarially Diversified Rehearsal Memory (ADRM): Mitigating Memory Overfitting Challenge in Continual Learning

TL;DR

This work tackles memory overfitting in rehearsal-based continual learning by introducing Adversarially Diversified Rehearsal Memory (ADRM), which uses one-step FGSM perturbations to diversify memory samples and strengthen model robustness to both natural and adversarial noise. ADRM interleaves adversarially diversified memory with current-task data and analyzes five diversification ratios, finding that a 10% diversification rate yields the best generalization-robustness trade-off. Empirical results on CIFAR-10, CIFAR-10-C, and adversarial CIFAR-10 demonstrate that ADRM improves robustness to noise and attacks while maintaining stable feature distributions; CNN representations become more continual-robust, as shown by t-SNE and CKA analyses, and feature disentanglement reveals more class-specific activations. Overall, ADRM offers a practical, scalable approach to mitigating rehearsal memory overfitting with significant implications for safety-critical continual learning applications; the authors provide public code for replication.

Abstract

Continual learning focuses on learning non-stationary data distribution without forgetting previous knowledge. Rehearsal-based approaches are commonly used to combat catastrophic forgetting. However, these approaches suffer from a problem called "rehearsal memory overfitting, " where the model becomes too specialized on limited memory samples and loses its ability to generalize effectively. As a result, the effectiveness of the rehearsal memory progressively decays, ultimately resulting in catastrophic forgetting of the learned tasks. We introduce the Adversarially Diversified Rehearsal Memory (ADRM) to address the memory overfitting challenge. This novel method is designed to enrich memory sample diversity and bolster resistance against natural and adversarial noise disruptions. ADRM employs the FGSM attacks to introduce adversarially modified memory samples, achieving two primary objectives: enhancing memory diversity and fostering a robust response to continual feature drifts in memory samples. Our contributions are as follows: Firstly, ADRM addresses overfitting in rehearsal memory by employing FGSM to diversify and increase the complexity of the memory buffer. Secondly, we demonstrate that ADRM mitigates memory overfitting and significantly improves the robustness of CL models, which is crucial for safety-critical applications. Finally, our detailed analysis of features and visualization demonstrates that ADRM mitigates feature drifts in CL memory samples, significantly reducing catastrophic forgetting and resulting in a more resilient CL model. Additionally, our in-depth t-SNE visualizations of feature distribution and the quantification of the feature similarity further enrich our understanding of feature representation in existing CL approaches. Our code is publically available at https://github.com/hikmatkhan/ADRM.

Figures (9)

• Figure 1: Idea illustration: t-SNE visualization showing the dispersion of 'airplane' class features under increasing adversarial perturbations (scaled as $\frac{\epsilon}{255}$). Dark blue dots represent the original airplane class samples, while progressively lighter shades of blue indicate the five levels of adversarially diversified samples. The visualization demonstrates how increasing the strength of the perturbation results in a divergence from the original class cluster. These adversarially diversified samples retain inherent airplane characteristics and can considered as potential rehearsal samples to enhance the diversity and complexity of the memory to prevent rehearsal memory overfitting. 1 row, the samples enclosed in a green rectangle represent the original, undiversified baseline memory. 2 row, the samples enclosed in an orange rectangle are adversarially diversified airplanes correctly classified by the CL model despite diversification. 3 row, the samples enclosed in a red rectangle are adversarially diversified airplanes incorrectly classified, highlighting potential areas for enhancing model robustness. The diversity in rehearsal samples is crucial to prevent memory overfitting and increase the complexity of memory samples, consequently improving the robustness of the CL model (best viewed in color).
• Figure 2: Examples of the six different severity levels for the 19 different noise types. Severity 0 represents the original clean image whereas severity 5 corresponds to the most severe corruption (Best viewed in color).
• Figure 3: Robustness comparison of eleven CL approaches against nineteen types of natural image corruptions. Each subplot corresponds to a different noise type, with the x-axis representing the severity level from 0 (no corruption) to 5 (maximum corruption) and the y-axis displaying the average accuracy of each model. The ADRM shows resilience and comparatively suffers less from catastrophic forgetting. It outperformed other models in 15 out of the 19 tested noise conditions, highlighted with a yellow rectangle in the graph (best viewed in color).
• Figure 4: Robustness comparison of fifteen CL approaches on the adversarially perturbed CIFAR-10 validation set khan2022adversarially. The x-axis represents the epsilon value ($\epsilon/255$), indicating the strength of the adversarial perturbation, while the y-axis shows the models' average accuracy. The sample images at the bottom are adversarially perturbed using the Fast Gradient Sign Method (FGSM) with increasing perturbation strength ($\epsilon$) from left to right. ADRM and its variants demonstrate enhanced adversarial robustness over a range of attack intensities relative to other CL approaches, indicating a stronger grasp of the fundamental class concepts within the dataset (best viewed in color).
• Figure 5: Adversarial examples created using PGD-$L_{\inf}$. On the $x$-axis, we can see the different adversarially perturbed images from the dataset. The different strengths of the level of the attacks ($\epsilon$) are plotted in the $y$-axis. The level of the attack ($\epsilon$) determines the strength of the adversarial attack; the higher the value, the stronger the adversarial attack and the more perturbed the input image (Best viewed in color).