ER-FSL: Experience Replay with Feature Subspace Learning for Online Continual Learning

TL;DR

This work tackles catastrophic forgetting in online continual learning by separating the learning and replay processes into different feature spaces. It introduces ER-FSL, which learns each new task in a dedicated feature subspace while replaying buffered past data in an accumulated, higher-dimensional space to preserve historical knowledge. The approach is validated across three image-classification benchmarks, showing consistent improvements over state-of-the-art replay-based methods, and ablations confirm the necessity of both subspace learning and accumulated-space replay. The method is efficient and practical, with public code available for reproducibility and further research.

Abstract

Online continual learning (OCL) involves deep neural networks retaining knowledge from old data while adapting to new data, which is accessible only once. A critical challenge in OCL is catastrophic forgetting, reflected in reduced model performance on old data. Existing replay-based methods mitigate forgetting by replaying buffered samples from old data and learning current samples of new data. In this work, we dissect existing methods and empirically discover that learning and replaying in the same feature space is not conducive to addressing the forgetting issue. Since the learned features associated with old data are readily changed by the features related to new data due to data imbalance, leading to the forgetting problem. Based on this observation, we intuitively explore learning and replaying in different feature spaces. Learning in a feature subspace is sufficient to capture novel knowledge from new data while replaying in a larger feature space provides more feature space to maintain historical knowledge from old data. To this end, we propose a novel OCL approach called experience replay with feature subspace learning (ER-FSL). Firstly, ER-FSL divides the entire feature space into multiple subspaces, with each subspace used to learn current samples. Moreover, it introduces a subspace reuse mechanism to address situations where no blank subspaces exist. Secondly, ER-FSL replays previous samples using an accumulated space comprising all learned subspaces. Extensive experiments on three datasets demonstrate the superiority of ER-FSL over various state-of-the-art methods.

Figures (6)

• Figure 1: The comparison of existing studies and our work. (a) An example of existing studies. Top: the model trains all samples in the same feature space; Bottom: the change of samples in the feature space using existing methods. (b) An example of our method. Top: different from existing studies, the model learns current samples in a feature subspace while replaying buffered samples in a larger one (e.g., feature whole-space). Bottom: the change of samples in the feature space using our approach.
• Figure 2: The analysis results on CIFAR10 with 2 learning tasks when the memory buffer size is 1000.
• Figure 3: The overview of our ER-FSL framework.
• Figure 4: The feature space of different tasks.
• Figure 5: Average accuracy rate on observed learning tasks on three datasets when the memory buffer size is 1000.