Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives

Qiushi Sun, Chengcheng Han, Nuo Chen, Renyu Zhu, Jingyang Gong, Xiang Li, Ming Gao

TL;DR

Open LMaaS settings restrict gradient access, motivating gradient-free prompt tuning. The authors propose BBT-RGB, a gradient-free framework with Three components: Two-Stage DFOs for robust convergence, M2 Verbalizers for leveraging multiple label cues, and In2 Initialization for strong prompt startup. Across seven GLUE-like tasks with a RoBERTa-Large backbone, BBT-RGB delivers strong gains over gradient-free baselines and even surpasses full fine-tuning on some entailment tasks, all with improved stability. The work offers a cost-efficient, plug-and-play pathway to adapt LLMs via prompts, accompanied by public code for broader adoption.

Abstract

Large language models (LLMs) have shown increasing power on various natural language processing (NLP) tasks. However, tuning these models for downstream tasks usually needs exorbitant costs or is unavailable due to commercial considerations. Recently, black-box tuning has been proposed to address this problem by optimizing task-specific prompts without accessing the gradients and hidden representations. However, most existing works have yet fully exploited the potential of gradient-free optimization under the scenario of few-shot learning. In this paper, we describe BBT-RGB, a suite of straightforward and complementary techniques for enhancing the efficiency and performance of black-box optimization. Specifically, our method includes three plug-and-play components: (1) Two-stage derivative-free optimization strategy that facilitates fast convergence and mitigates overfitting; (2) Automatic verbalizer construction with its novel usage under few-shot settings; (3) Better prompt initialization policy based on instruction search and auto-selected demonstration. Extensive experiments across various tasks on natural language understanding and inference demonstrate the effectiveness of our method. Our codes are publicly available at https://github.com/QiushiSun/BBT-RGB.

Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives

TL;DR

Open LMaaS settings restrict gradient access, motivating gradient-free prompt tuning. The authors propose BBT-RGB, a gradient-free framework with Three components: Two-Stage DFOs for robust convergence, M2 Verbalizers for leveraging multiple label cues, and In2 Initialization for strong prompt startup. Across seven GLUE-like tasks with a RoBERTa-Large backbone, BBT-RGB delivers strong gains over gradient-free baselines and even surpasses full fine-tuning on some entailment tasks, all with improved stability. The work offers a cost-efficient, plug-and-play pathway to adapt LLMs via prompts, accompanied by public code for broader adoption.

Abstract

Large language models (LLMs) have shown increasing power on various natural language processing (NLP) tasks. However, tuning these models for downstream tasks usually needs exorbitant costs or is unavailable due to commercial considerations. Recently, black-box tuning has been proposed to address this problem by optimizing task-specific prompts without accessing the gradients and hidden representations. However, most existing works have yet fully exploited the potential of gradient-free optimization under the scenario of few-shot learning. In this paper, we describe BBT-RGB, a suite of straightforward and complementary techniques for enhancing the efficiency and performance of black-box optimization. Specifically, our method includes three plug-and-play components: (1) Two-stage derivative-free optimization strategy that facilitates fast convergence and mitigates overfitting; (2) Automatic verbalizer construction with its novel usage under few-shot settings; (3) Better prompt initialization policy based on instruction search and auto-selected demonstration. Extensive experiments across various tasks on natural language understanding and inference demonstrate the effectiveness of our method. Our codes are publicly available at https://github.com/QiushiSun/BBT-RGB.
Paper Structure (31 sections, 1 equation, 4 figures, 6 tables, 2 algorithms)

This paper contains 31 sections, 1 equation, 4 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Large Language Models and APIs
  4. Prompt-based Learning
  5. Derivative-free Optimization
  6. BBT-RGB
  7. Two-Stage DFOs
  8. M2 Verbalizers
  9. In2 Initialization
  10. Experiments
  11. Experimental Settings
  12. Backbone
  13. Datasets
  14. Methods and Hyperparameters
  15. Main Results
  16. ...and 16 more sections

Figures (4)

  • Figure 1: An illustration of BBT-RGB. Given a backbone model with $L$ layers. The target is to optimize continuous prompts $z^{l}, l\in [1, L]$. We use Red, Green and Blue to indicate three distinct aspects of our strategy, which inspired the naming of our method. M2 Verbalizers (Multi-Mixed Verbalizers) further utilize the information provided by the LLMs. In2 Initialization (Instruction learning + In-context learning) improves prompt-based tuning by integrating both instruction and demonstration, noted as $p_{l }$. And Two-Stage DFOs exploit the advantages of different optimization methods. represents the combination of derivative-free optimizers. (Best viewed in color.)
  • Figure 2: Comparing BBT-RGB with other tuning methods on average performance over seven tasks described in section \ref{['exp:settting']}. The size of the circle is proportional to the standard deviation.
  • Figure 3: Comparison of original CMA-ES and Two-stage DFOs on Yelp dataset.
  • Figure 4: An illustration of Two-Stage DFOs on SST-2