Delve into Base-Novel Confusion: Redundancy Exploration for Few-Shot Class-Incremental Learning

TL;DR

A method is proposed named Redundancy Decoupling and Integration (RDI), which first decouples redundancies from base-class space to shrink the intra-base-class feature space and integrates the redundancies as a dummy class to enlarge the inter-base-class feature space.

Abstract

Few-shot class-incremental learning (FSCIL) aims to acquire knowledge from novel classes with limited samples while retaining information about base classes. Existing methods address catastrophic forgetting and overfitting by freezing the feature extractor during novel-class learning. However, these methods usually tend to cause the confusion between base and novel classes, i.e., classifying novel-class samples into base classes. In this paper, we delve into this phenomenon to study its cause and solution. We first interpret the confusion as the collision between the novel-class and the base-class region in the feature space. Then, we find the collision is caused by the label-irrelevant redundancies within the base-class feature and pixel space. Through qualitative and quantitative experiments, we identify this redundancy as the shortcut in the base-class training, which can be decoupled to alleviate the collision. Based on this analysis, to alleviate the collision between base and novel classes, we propose a method for FSCIL named Redundancy Decoupling and Integration (RDI). RDI first decouples redundancies from base-class space to shrink the intra-base-class feature space. Then, it integrates the redundancies as a dummy class to enlarge the inter-base-class feature space. This process effectively compresses the base-class feature space, creating buffer space for novel classes and alleviating the model's confusion between the base and novel classes. Extensive experiments across benchmark datasets, including CIFAR-100, miniImageNet, and CUB-200-2011 demonstrate that our method achieves state-of-the-art performance.

Figures (11)

• Figure 1: (a) FSCIL entails mastering base classes through ample examples and incrementally addressing novel classes with few samples. (b) Left: Performance on base classes, novel classes, and all classes. BA-Acc, NA-Acc, and AA-Acc denote the average accuracy of classifying Base-class, Novel-class, and All-class samples against All encountered classes, respectively. B- represents using the Baseline fixed-backbone method, while O- represents using Ours. Right: Performance on novel-class samples. NN-Acc denotes the average accuracy of classifying Novel-class samples against only encountered Novel classes. We can see the model tends to classify novel-class samples into base classes, which is a major cause of the low performance in FSCIL. We term this problem as the confusion between novel and base classes, which we aim to interpret and alleviate in this paper. (c) Intuitive interpretation of the confusion in the feature space and the pixel space.
• Figure 2: (a) Feature distribution and decision boundary of six base classes. (b) Feature distribution of six introduced novel classes, based on the six existing base classes. The introduced novel classes demonstrate a potential for confusion with the base classes.
• Figure 3: Because of the misalignment between decreasing $\sum_{m\neq c}^{}e^{\tau cos(x_{i},w_{m} )}$ and increasing $e^{\tau cos(x_{i},w_{c} )}$, the ordinary cross-entropy loss could push the model to learn redundant features that are both distant from other classes and the class $m$.
• Figure 4: Decoupling of feature map into central and redundant patches. The redundant patches correspond to label-irrelevant objects in the base-class pixel space.
• Figure 5: (a) Visualization of the distribution of ALR and ALI features from base classes. Pixel-space Redundancies (ALI features) imply redundancy in feature space and enlarge the base-class space. (b) Visualization of the distribution of newly introduced novel classes, based on the existing ALR and ALI features. The gray areas indicate regions of base-class redundancies. We can see novel classes are located in this gray region, indicating that redundancy in the base-class feature space leads to confusion.