Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning

TL;DR

Q-tuning introduces a queue-based prompt tuning framework for lifelong few-shot learning, maintaining a finite Q-prompt memory to support continual tasks without data replay. It couples a low-rank knowledge-aggregation mechanism over past prompts with a PCA-based De-Q eviction and a globally shared prefix prompt equipped with memory retention to preserve prior knowledge. Empirical results across short, long, and lifelong task sequences show Q-tuning outperforms state-of-the-art continual prompt tuning and traditional prompt-tuning baselines, achieving scalable, near-constant time training and inference. Limitations include absence of backward transfer and the need for task identities at test time, motivating future work on unknown-task inference and retrieval-based prompt selection.

Abstract

This paper introduces \textbf{Q-tuning}, a novel approach for continual prompt tuning that enables the lifelong learning of a pre-trained language model. When learning a new task, Q-tuning trains a task-specific prompt by adding it to a prompt queue consisting of the prompts from older tasks. To better transfer the knowledge of old tasks, we design an adaptive knowledge aggregation technique that reweighs previous prompts in the queue with a learnable low-rank matrix. Once the prompt queue reaches its maximum capacity, we leverage a PCA-based eviction rule to reduce the queue's size, allowing the newly trained prompt to be added while preserving the primary knowledge of old tasks. In order to mitigate the accumulation of information loss caused by the eviction, we additionally propose a globally shared prefix prompt and a memory retention regularization based on information theory. Extensive experiments demonstrate that our approach outperforms the state-of-the-art methods substantially on continual prompt tuning benchmarks. Moreover, our approach enables lifelong learning on linearly growing task sequences while requiring constant complexity for training and inference.

Figures (22)

• Figure 1: The overall framework of the proposed Q-tuning technology. Given a continually growing-up task sequence, we propose a prompt queue (Q-prompt) and a globally shared prefix prompt $\theta^i_{\mathcal{P}^{\ast}}$ to achieve the forward knowledge transfer, where the superscript of $\theta^i_{\mathcal{P}^{\ast}}$ denotes the $i$-th status. Moreover, we adopt a knowledge aggregation method to adaptively adjust the contribution of each fixed prompt $[\theta^1_{\mathcal{P}},\theta^2_{\mathcal{P}},\ldots,\theta^{i-1}_{\mathcal{P}}]$ in Q-prompt by using a rank-one matrix $\mathcal{W}^i$. We parameterize the trainable soft prompt by a two-layer residual MLP. If the length of the Q-prompt exceeds the limit, we apply a De-Q rule to discard less informative prompts in the queue.
• Figure 2: Visualization of aggregation matrix.
• Figure 3: Forward transfer score of different approaches on the order 8 (20 samples/class).
• Figure 4: Forward knowledge transfer results of Order 9 using 20 samples/class. Results are averaged over 3 runs.
• Figure 4: Forward transfer score of different approaches on the order 8 (200 samples/class).

Theorems & Definitions (2)

• Proposition 1
• proof