PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning

TL;DR

This work tackles class-incremental learning (CIL) with pre-trained models by addressing catastrophic forgetting through an input-agnostic, global prompt framework named PEARL. PEARL combines Sequential Prompt Adaptation (SPA) and Negative-feedback Knowledge Accumulation (NKA), where a momentum-based update governed by $\alpha^{\tau}=\gamma\alpha^{\tau-1}+(1-\gamma)\sigma(\mathrm{mae})$ modulates how knowledge is integrated into memory prompts $ {[PT]}^{mem}$. The backbone is frozen ViT, while a learnable prompt encoder $\bf E$ and prompt pool $\bf P$ generate a shared prompt that evolves across sessions; a Segmented Positional Encoding (SPE) encodes task information to support cross-task learning. Experiments on six benchmarks with ViT backbones show state-of-the-art performance, with substantial gains over prior prompt-based, representation-based, and model-mixture approaches, and analyses reveal that NKA captures task correlations and stabilizes continual learning.

Abstract

Class-incremental learning (CIL) aims to continuously introduce novel categories into a classification system without forgetting previously learned ones, thus adapting to evolving data distributions. Researchers are currently focusing on leveraging the rich semantic information of pre-trained models (PTMs) in CIL tasks. Prompt learning has been adopted in CIL for its ability to adjust data distribution to better align with pre-trained knowledge. This paper critically examines the limitations of existing methods from the perspective of prompt learning, which heavily rely on input information. To address this issue, we propose a novel PTM-based CIL method called Input-Agnostic Prompt Enhancement with Negative Feedback Regulation (PEARL). In PEARL, we implement an input-agnostic global prompt coupled with an adaptive momentum update strategy to reduce the model's dependency on data distribution, thereby effectively mitigating catastrophic forgetting. Guided by negative feedback regulation, this adaptive momentum update addresses the parameter sensitivity inherent in fixed-weight momentum updates. Furthermore, it fosters the continuous enhancement of the prompt for new tasks by harnessing correlations between different tasks in CIL. Experiments on six benchmarks demonstrate that our method achieves state-of-the-art performance. The code is available at: https://github.com/qinyongchun/PEARL.

Figures (5)

• Figure 1: The comparison of (a) input-dependent prompt and (b) input-independent prompt. L2P and DualPrompt following the paradigm in (a), select the best matched prompts, while CODA-Prompt assemble the prompt pool with learnable components during the "select" phase.
• Figure 2: The illustration of the proposed PEARL. The ViT consists of 12 blocks, with a $L$-layer prompt encoder added to the last few blocks of the ViT. The ViT is frozen during training, while the prompt encoder and prompt pool remain learnable.
• Figure 3: The illustration of the proposed NKA mechanism. A low $mae$ indicates good knowledge retention, enabling the model to focus more on the new task.
• Figure 4: The curves of $mae$ and $\alpha$, across different initial value of $\alpha^0$. Results are derived from the second session of CUB.
• Figure 5: The value of $\alpha^{\mathrm{final}}$ across different settings.