CASP: Few-Shot Class-Incremental Learning with CLS Token Attention Steering Prompts

TL;DR

Few-shot class-incremental learning (FSCIL) requires rapidly incorporating new classes from scarce data while avoiding catastrophic forgetting of earlier classes. The authors present CASP, a prompt-based framework that steers the Vision Transformer's CLS token via symmetric prompt injections, attentional perturbation, CLS-domain adaptation, and Manifold Token Mixup to produce domain-generalizable representations without incremental fine-tuning. Empirical results on CUB200, CIFAR100, and ImageNet-R show state-of-the-art performance with dramatically reduced parameter overhead, validating improved generalization and robustness. The work offers a practical, efficient route to continual learning in vision, with potential extensions to multimodal FSCIL and related transfer-learning contexts.

Abstract

Few-shot class-incremental learning (FSCIL) presents a core challenge in continual learning, requiring models to rapidly adapt to new classes with very limited samples while mitigating catastrophic forgetting. Recent prompt-based methods, which integrate pretrained backbones with task-specific prompts, have made notable progress. However, under extreme few-shot incremental settings, the model's ability to transfer and generalize becomes critical, and it is thus essential to leverage pretrained knowledge to learn feature representations that can be shared across future categories during the base session. Inspired by the mechanism of the CLS token, which is similar to human attention and progressively filters out task-irrelevant information, we propose the CLS Token Attention Steering Prompts (CASP). This approach introduces class-shared trainable bias parameters into the query, key, and value projections of the CLS token to explicitly modulate the self-attention weights. To further enhance generalization, we also design an attention perturbation strategy and perform Manifold Token Mixup in the shallow feature space, synthesizing potential new class features to improve generalization and reserve the representation capacity for upcoming tasks. Experiments on the CUB200, CIFAR100, and ImageNet-R datasets demonstrate that CASP outperforms state-of-the-art methods in both standard and fine-grained FSCIL settings without requiring fine-tuning during incremental phases and while significantly reducing the parameter overhead.

Figures (7)

• Figure 1: Comparison of two continual learning paradigms. In the continual fine-tuning paradigm, the model parameters are updated in each incremental session. In the incremental-session freezing paradigm, only the base session involves model training, while the parameters are frozen in subsequent incremental sessions, requiring no updates.
• Figure 2: CASP Architecture. The CASP framework introduces class-shared bias parameters into CLS token projections and employs feature space Mixup, enabling dynamic self-attention adjustment and generalization without full fine-tuning.
• Figure 3: SOTA comparisons on the CIFAR100, ImageNet-R, and CUB200 benchmarks.
• Figure 4: Sensitivity study of the Manifold Token Mixup hyperparameter $\lambda_{mix}$, illustrating the model's $A_{L}$ across varying values of $\lambda_{mix}$.
• Figure 5: Performance trend of $A_L$ plotted against the depth of the layer where the MTM is applied.