Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards

TL;DR

Clip-Tuning tackles the challenge of derivative-free prompt learning in few-shot NLP by leveraging an ensemble of frozen, deterministically clipped subnetworks to generate diverse reward signals. Predictions from these thinned subnetworks are aggregated to guide CMA-ES-based prompt optimization, enabling effective learning without backpropagation. The approach yields state-of-the-art performance among gradient-free methods and rivals gradient-based prompt tuning on seven NL benchmarks, with practical advantages for API-based, inference-only PLMs. The work highlights the value of reward diversity and over-parameterization in enabling efficient, low-cost prompt optimization in real-world deployment scenarios.

Abstract

Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did not take full advantage of the over-parameterized characteristics of large pre-trained language models (PLMs). In this paper, we propose Clip-Tuning, a simple yet effective method that adopts diverse frozen "thinned" networks of PLMs to obtain a mixture of rewards and thus advance the derivative-free prompt learning. The thinned networks consist of all the hidden units that survive a stationary dropout strategy, whose inference predictions reflect an ensemble of partial views over prompted training samples. Our method outperforms previous gradient-free prompt learning methods and achieves parity with gradient-based counterparts on seven language understanding benchmarks under few-shot settings.

Figures (4)

• Figure 1: Illustration of Clip-Tuning process.
• Figure 2: Varying clip ratios vs. performance.
• Figure 3: (a) Varying subnetworks ($N$) vs. performance. (b) Dynamic / no / static clipping vs. performance. (c) Size of training samples (SST-2).
• Figure 4: Different prompt initialization vs. performance (SST-2).