PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning

TL;DR

This paper identifies two fundamental causes of forgetting in CIL: representation bias and classifier bias and proposes a simple yet effective dual-bias reduction framework, which leverages self-supervised transformation (SST) in the input space and prototype augmentation (protoAug) in the feature space.

Abstract

Class-incremental learning (CIL) aims to recognize new classes incrementally while maintaining the discriminability of old classes. Most existing CIL methods are exemplar-based, i.e., storing a part of old data for retraining. Without relearning old data, those methods suffer from catastrophic forgetting. In this paper, we figure out two inherent problems in CIL, i.e., representation bias and classifier bias, that cause catastrophic forgetting of old knowledge. To address these two biases, we present a simple and novel dual bias reduction framework that employs self-supervised transformation (SST) in input space and prototype augmentation (protoAug) in deep feature space. On the one hand, SST alleviates the representation bias by learning generic and diverse representations that can transfer across different tasks. On the other hand, protoAug overcomes the classifier bias by explicitly or implicitly augmenting prototypes of old classes in the deep feature space, which poses tighter constraints to maintain previously learned decision boundaries. We further propose hardness-aware prototype augmentation and multi-view ensemble strategies, leading to significant improvements. The proposed framework can be easily integrated with pre-trained models. Without storing any samples of old classes, our method can perform comparably with state-of-the-art exemplar-based approaches which store plenty of old data. We hope to draw the attention of researchers back to non-exemplar CIL by rethinking the necessity of storing old samples in CIL.

Figures (16)

• Figure 1: Left: In non-exemplar CIL, the model is updated continuously on new classes without accessing old data. Middle: after learning new classes, previous learned decision boundaries are distorted and the representation confusion is severe. Right: illustration of representation and classifier bias between old and new classes.
• Figure 2: Illustration of the proposed dual bias reduction framework. Classes of the current task are augmented by rotation based transformation. In the deep feature space, we augment the memorized prototypes explicitly (directly generate augmented feature instances) or implicitly (transforming the original sampling process to a regularization term). A hardness-aware (informative) protoAug strategy is further proposed to compensate for the original protoAug.
• Figure 3: Illustration of hardness-aware prototype augmentation. The constructed hard feature instances provide compensable information to refine the estimated distribution.
• Figure 4: TSNE van2008visualizing visualization of class representations in the feature space when learning MNIST LeCun2005TheMD incrementally. The outputted features are 2-dimensional which is suitable for visualization. Best viewed in color.
• Figure 5: TSNE van2008visualizing visualization shows that SST improves the separation of novel classes, reducing the overlap between base and novel classes.