Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Prompt Adjustment for Multi-Label Class-Incremental Learning

Haifeng Zhao, Yuguang Jin, Leilei Ma

TL;DR

This work tackles Multi-Label Class-Incremental Learning (MLCIL) by leveraging CLIP-based prompt tuning and introducing two key components: Incremental Context Prompting (ICP), which uses category-specific and contextual prompts to balance representations across growing label sets, and Selective Confidence Cluster Replay (SCCR), which replays diverse, high-utility samples selected via clustering and model confidence. A Textual Prompt Consistency Loss further stabilizes prompt evolution across incremental steps. Empirical results on MS-COCO and PASCAL VOC demonstrate substantial gains over prior methods, with performance approaching joint training under larger replay buffers, validating the effectiveness of image-text matching for MLCIL and the efficiency of the proposed framework. The approach offers a scalable, memory-efficient path for continual multi-label classification in dynamic environments.

Abstract

Significant advancements have been made in single label incremental learning (SLCIL),yet the more practical and challenging multi label class incremental learning (MLCIL) remains understudied. Recently,visual language models such as CLIP have achieved good results in classification tasks. However,directly using CLIP to solve MLCIL issue can lead to catastrophic forgetting. To tackle this issue, we integrate an improved data replay mechanism and prompt loss to curb knowledge forgetting. Specifically,our model enhances the prompt information to better adapt to multi-label classification tasks and employs confidence-based replay strategy to select representative samples. Moreover, the prompt loss significantly reduces the model's forgetting of previous knowledge. Experimental results demonstrate that our method has substantially improved the performance of MLCIL tasks across multiple benchmark datasets,validating its effectiveness.

Dynamic Prompt Adjustment for Multi-Label Class-Incremental Learning

TL;DR

This work tackles Multi-Label Class-Incremental Learning (MLCIL) by leveraging CLIP-based prompt tuning and introducing two key components: Incremental Context Prompting (ICP), which uses category-specific and contextual prompts to balance representations across growing label sets, and Selective Confidence Cluster Replay (SCCR), which replays diverse, high-utility samples selected via clustering and model confidence. A Textual Prompt Consistency Loss further stabilizes prompt evolution across incremental steps. Empirical results on MS-COCO and PASCAL VOC demonstrate substantial gains over prior methods, with performance approaching joint training under larger replay buffers, validating the effectiveness of image-text matching for MLCIL and the efficiency of the proposed framework. The approach offers a scalable, memory-efficient path for continual multi-label classification in dynamic environments.

Abstract

Significant advancements have been made in single label incremental learning (SLCIL),yet the more practical and challenging multi label class incremental learning (MLCIL) remains understudied. Recently,visual language models such as CLIP have achieved good results in classification tasks. However,directly using CLIP to solve MLCIL issue can lead to catastrophic forgetting. To tackle this issue, we integrate an improved data replay mechanism and prompt loss to curb knowledge forgetting. Specifically,our model enhances the prompt information to better adapt to multi-label classification tasks and employs confidence-based replay strategy to select representative samples. Moreover, the prompt loss significantly reduces the model's forgetting of previous knowledge. Experimental results demonstrate that our method has substantially improved the performance of MLCIL tasks across multiple benchmark datasets,validating its effectiveness.
Paper Structure (20 sections, 8 equations, 3 figures, 4 tables)

This paper contains 20 sections, 8 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Single-Label Class-Incremental Learning
  4. Multi-Label Class-Incremental Learning
  5. Prompt Tuning for Incremental Learning
  6. Methodology
  7. Framework
  8. Incremental Context Prompting Learner
  9. Selective Confidence Cluster Replay
  10. Optimization Objective
  11. Evaluation
  12. Experiment setup
  13. Comparsion results
  14. Qualitative results
  15. Ablation Studies
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Illustration of incremental learning for both single-label and multi-label classification tasks. It is assumed that three classes are to be learned, with models $\theta_{1}$, $\mathbf{\theta}_{2}$, and $\mathbf{\theta}_{3}$ being trained in consecutive sessions.
  • Figure 2: Overview of our framework.
  • Figure 3: Visualization of the attention maps on MS-COCO under B40-C10 setting.