P2L-CA: An Effective Parameter Tuning Framework for Rehearsal-Free Multi-Label Class-Incremental Learning

TL;DR

This work tackles Multi-Label Class-Incremental Learning under a strict rehearsal-free constraint by introducing P2L-CA, a parameter-efficient framework that combines a class-specific Prompt-to-Label module with a Continuous Adapter module. The P2L component decouples multi-label representations through per-class prompts and linguistic priors, while the CA component bridges domain gaps with adapters embedded in a frozen ViT backbone, enabling rapid adaptation to new classes. The approach achieves state-of-the-art performance on standard benchmarks (MS-COCO and PASCAL VOC) and maintains strong generalization in CIL, including challenging long-sequence settings, without memory buffers; CLIP-based semantic initialization further boosts performance (P2L-CA+). Overall, P2L-CA provides a scalable, privacy-preserving, and efficient solution for continual multi-label recognition with applications in open-world and real-world systems.

Abstract

Multi-label Class-Incremental Learning aims to continuously recognize novel categories in complex scenes where multiple objects co-occur. However, existing approaches often incur high computational costs due to full-parameter fine-tuning and substantial storage overhead from memory buffers, or they struggle to address feature confusion and domain discrepancies adequately. To overcome these limitations, we introduce P2L-CA, a parameter-efficient framework that integrates a Prompt-to-Label module with a Continuous Adapter module. The P2L module leverages class-specific prompts to disentangle multi-label representations while incorporating linguistic priors to enforce stable semantic-visual alignment. Meanwhile, the CA module employs lightweight adapters to mitigate domain gaps between pre-trained models and downstream tasks, thereby enhancing model plasticity. Extensive experiments across standard and challenging MLCIL settings on MS-COCO and PASCAL VOC show that P2L-CA not only achieves substantial improvements over state-of-the-art methods but also demonstrates strong generalization in CIL scenarios, all while requiring minimal trainable parameters and eliminating the need for memory buffers.

Figures (5)

• Figure 1: Effectiveness of the P2L-CA Framework: (a) We present the t-SNE visualizations of 20 classes in COCO, illustrating that our method effectively addresses feature confusion. (b) We highlight that the CA module in P2L-CA effectively bridges the domain gap, leading to superior transfer ability and resulting in higher mAP scores for new classes.
• Figure 2: The Framework of our proposed P2L-CA. The image tokens are obtained from the first $k$ layers of the image encoder. Then they are concatenated with the prompt corresponding to each category and passed jointly to the subsequent encoder layers for self-attention computation. The prompts can be derived from a class-specific prompt pool with random initialization or a semantic feature initialized by a language model. The prediction is obtained by feeding the prompts part of the encoder output into the FFN classifiers.
• Figure 3: (a) The illustration of the Continual-Adapter module in the Self-Attention Block. (b) Illustration of the adapter structure and the novel training paradigm at various stages.
• Figure 4: Influence of the layers of equipped adapters and inserted prompts
• Figure 5: t-SNE visualization of prompt tokens in the feature space