Learning to Prompt for Continual Learning

TL;DR

Learning to Prompt (L2P) reframes continual learning by storing knowledge in a compact, learnable prompt memory rather than updating the entire model or buffering past data. A frozen pre-trained backbone is augmented with a prompt pool and an instance-wise query mechanism that selects task-relevant prompts without needing task identity at test time. The method decouples shared and task-specific knowledge, enabling strong performance across class-incremental, domain-incremental, and task-agnostic settings while reducing reliance on rehearsal buffers. Empirical results across benchmarks show L2P surpasses prior state-of-the-art in many scenarios and remains competitive even without rehearsal, highlighting the practical potential for memory-efficient continual learning with prompt-based conditioning.

Abstract

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.

Figures (4)

• Figure 1: Overview of the L2P framework. Compared with typical methods that adapt entire or partial model weights to tasks sequentially with a rehearsal buffer to avoid forgetting, L2P uses a single backbone model and learns a prompt pool to instruct the model conditionally. Task-specific knowledge is stored inside a prompt pool, thus a rehearsal buffer is no longer mandatory to mitigate forgetting.L2P automatically selects and updates prompts from the pool in an instance-wise fashion, thus task identity is not required at test time.Notably, our largest prompt space is smaller than the size of one $224\times224$ image.
• Figure 2: Illustration of L2P at test time. We follow the same procedure at training time: First, L2P selects a subset of prompts from a key-value paired prompt pool based on our proposed instance-wise query mechanism. Then, L2P prepends the selected prompts to the input tokens. Finally, L2P feeds the extended tokens to the model, and optimize the prompt pool through the loss defined in equation \ref{['eq:full_loss']}. The objective is learning to select and update prompts to instruct the prediction of the pre-trained backbone model.
• Figure 3: Prompt selection histograms for (left) Split CIFAR-100 and (right) 5-datasets. CIFAR-100 has higher intra-task similarity compared to 5-datasets, thus largely sharing prompts between tasks results in good performance, while 5-datasets favors more task-specific prompts. We only show the first 5 tasks for Split CIFAR-100 for better readability.
• Figure 4: Left-Middle: Average accuracy w.r.t prompt length $L_p$ and prompt selection size $N$ for Split CIFAR-100 and 5-datasets, respectively, given $M=20$. Right: Average accuracy (%) w.r.t. prompt pool size $M$, given $L_p=5$, $N=5$ for Split CIFAR-100 and $L_p=5$, $N=4$ for 5-datasets.