Black-box Prompt Tuning with Subspace Learning

TL;DR

This paper tackles the limited cross-task versatility of black-box prompt tuning by introducing Black-box Prompt Tuning with Subspace Learning (BSL). BSL first identifies common low-dimensional subspaces across related source tasks via a meta-learning procedure on deep continuous prompts, then leverages derivative-free optimization to tune prompts within a selected subspace for a target task. Empirically, BSL achieves competitive results across text classification, QE, and generation tasks and shows improvements over existing black-box methods while approaching derivative-based methods, with notable gains in efficiency and convergence speed. The approach is particularly suitable for LLM service scenarios, offering a reusable subspace-based customization paradigm that reduces computation and API calls while maintaining high task performance.

Abstract

Black-box prompt tuning employs derivative-free optimization algorithms to learn prompts within low-dimensional subspaces rather than back-propagating through the network of Large Language Models (LLMs). Recent studies reveal that black-box prompt tuning lacks versatility across tasks and LLMs, which we believe is related to the suboptimal choice of subspaces. In this paper, we introduce Black-box prompt tuning with Subspace Learning (BSL) to enhance the versatility of black-box prompt tuning. Based on the assumption that nearly optimal prompts for similar tasks reside in a common subspace, we propose identifying such subspaces through meta-learning on a collection of similar source tasks. Consequently, for a target task that shares similarities with the source tasks, we expect that optimizing within the identified subspace can yield a prompt that performs well on the target task. Experimental results confirm that our BSL framework consistently achieves competitive performance across various downstream tasks and LLMs.

Figures (7)

• Figure 1: In our BSL framework, deep continuous prompts are optimized within a selected low-dimensional subspace via reparameterization and are learned using derivative-free optimization (DFO) algorithms. We begin by identifying common low-dimensional subspaces that contain satisfactory solutions for source tasks through meta-learning. Subsequently, the identified subspace is utilized for black-box prompt tuning on target tasks that exhibit similarities to the source tasks.
• Figure 2: The trajectory of the subspace parameters during training is represented by the black line, with each point indicating a different subspace. We use meta-learning to identify a common low-dimensional subspace that contains prompts effective for the three source tasks.
• Figure 3: Learning curves of different black-box prompt tuning methods. The results are evaluated on a separate development set with 1,024 examples.
• Figure 4: Effect of source tasks on BSL with regard to task similarities.
• Figure 5: Effect of source tasks on BSL with regard to the number of source tasks.