PrePrompt: Predictive prompting for class incremental learning

TL;DR

PrePrompt introduces a predictive-prompting paradigm for class-incremental learning that replaces correlation-based key-value prompt retrieval with a two-stage prediction: first predict a task-specific prompt, then predict the label using that prompt. By leveraging a pre-trained vision transformer and a feature-translation mechanism, it balances stability and plasticity to mitigate forgetting across sequential tasks. The method achieves state-of-the-art results on multiple benchmarks with significantly fewer trainable parameters than prior prompt-based CIL approaches, and ablation studies confirm the value of its components. This work advances efficient continual learning by decoupling task-specific prompt prediction from label prediction and aligning old and new feature spaces, offering practical benefits for scalable deployment on resource-constrained devices.

Abstract

Class Incremental Learning (CIL) based on pre-trained models offers a promising direction for open-world continual learning. Existing methods typically rely on correlation-based strategies, where an image's classification feature is used as a query to retrieve the most related key prompts and select the corresponding value prompts for training. However, these approaches face an inherent limitation: fitting the entire feature space of all tasks with only a few trainable prompts is fundamentally challenging. We propose Predictive Prompting (PrePrompt), a novel CIL framework that circumvents correlation-based limitations by leveraging pre-trained models' natural classification ability to predict task-specific prompts. Specifically, PrePrompt decomposes CIL into a two-stage prediction framework: task-specific prompt prediction followed by label prediction. While theoretically appealing, this framework risks bias toward recent classes due to missing historical data for older classifier calibration. PrePrompt then mitigates this by incorporating feature translation, dynamically balancing stability and plasticity. Experiments across multiple benchmarks demonstrate PrePrompt's superiority over state-of-the-art prompt-based CIL methods. Code available at \href{github.com/libo-huang/preprompt}{github.com/libo-huang/preprompt}.

Figures (9)

• Figure 1: Comparison of existing prompt-based CIL methods and our PrePrompt approach. Existing prompt-based CIL methods typically utilize prompts as key-value pairs (i.e., $[$$]$) and select prompts based on feature-key correlation. Specifically, L2P employs k-nearest neighbors for selection, DualPrompt introduces a set of general-purpose prompts (i.e., $[$$]$), and CODA-Prompt combines all prompts with weights determined by their correlation. In contrast, the proposed PrePrompt simplifies this structure by using a single value prompt and selects the final prompt based on the initial prediction.
• Figure 2: T-SNE visualization of feature and prompt distributions on CIFAR-100. The plot compares key-prompt embeddings (large stars, color-coded by task) and feature embeddings (small dots) across four sequential tasks, each containing 25 classes. Since PrePrompt does not use key-prompts, we visualize the class feature means instead, where the class IDs are inferred by the prompt classifier. L2P, DualPrompt, and CODA-Prompt exhibit spatial overlap and clustering in their key-prompt embeddings (with some regions zoomed out), indicating inter-task interference. PrePrompt’s embeddings remain well-separated and strategically distributed in distinct regions. This highlights PrePrompt’s superior task-specific decoupling and robustness against catastrophic forgetting in prompt-based class incremental learning.
• Figure 3: (a) The PrePrompt framework operates through two sequential prediction stages: prompt prediction and label prediction. During prompt prediction, a pre-trained ViT is utilized to train a prompt classifier, which then predicts task-specific prompts for training a new label classifier to perform final image classification. Both stages leverage (b) feature translation, where the nearest new prototype is identified for each retained old prototype from prior tasks. The old features are constructed by transferring features from their nearest new prototype, followed by alignment to the original old prototype.
• Figure 4: Results of various prototype quantities.
• Figure 5: Results of various prompt lengths.

Theorems & Definitions (1)

• proof