Few-Shot Class Incremental Learning via Robust Transformer Approach

TL;DR

This work tackles Few-Shot Class-Incremental Learning under data scarcity and catastrophic forgetting by introducing ROBUSTA, a Robust Transformer Approach built on a Compact Convolution Transformer backbone. It integrates a stochastic classifier, batch normalization, delta-prefix parameters, non-parametric task inference, and prototype rectification to address overfitting, forgetting, and intra-class bias. The method achieves state-of-the-art results across CIFAR-100, Mini-ImageNet, and CUB-200-2011 without data augmentation, and ablation analyses validate the necessity of each component. The findings demonstrate that transformer-based architectures with careful task-aware adaptations can substantially improve continual learning performance in few-shot regimes, with practical implications for scalable, data-efficient lifelong learners.

Abstract

Few-Shot Class-Incremental Learning presents an extension of the Class Incremental Learning problem where a model is faced with the problem of data scarcity while addressing the catastrophic forgetting problem. This problem remains an open problem because all recent works are built upon the convolutional neural networks performing sub-optimally compared to the transformer approaches. Our paper presents Robust Transformer Approach built upon the Compact Convolution Transformer. The issue of overfitting due to few samples is overcome with the notion of the stochastic classifier, where the classifier's weights are sampled from a distribution with mean and variance vectors, thus increasing the likelihood of correct classifications, and the batch-norm layer to stabilize the training process. The issue of CF is dealt with the idea of delta parameters, small task-specific trainable parameters while keeping the backbone networks frozen. A non-parametric approach is developed to infer the delta parameters for the model's predictions. The prototype rectification approach is applied to avoid biased prototype calculations due to the issue of data scarcity. The advantage of ROBUSTA is demonstrated through a series of experiments in the benchmark problems where it is capable of outperforming prior arts with big margins without any data augmentation protocols.

Figures (9)

• Figure 1: Illustration of Stochastic Classifier
• Figure 2: Overview of the student model. We replace the LayerNorm layers in the model with layers.
• Figure 3: The blocks of transformers. We replace the LayerNorm layers in the transformer blocks. Moreover, we add a layer between the two linear layers of the .
• Figure 4: The trace of the accuracy of the studied methods on the three datasets under moderate base classes (60,60,100)
• Figure 5: The trace of the accuracy of the studied methods on the three datasets with small base classes (20,20,50)