Controllable Natural Language Generation with Contrastive Prefixes

TL;DR

The paper introduces a prefix-based framework for controllable NLG that keeps the language model frozen and learns small, attribute-specific prefix vectors. By training multiple prefixes jointly with supervised, semi-supervised, and unsupervised objectives, the approach captures inter-attribute relationships to enable single- and multi-aspect control while preserving linguistic quality and maintaining fast inference. Empirical results across sentiment, detoxification, and topic tasks show strong attribute alignment with competitive or superior quality compared with baselines like GPT-2, PPLM, and GeDi; multi-aspect control is achieved via concatenation or joint training. The work highlights the importance of a discriminative loss and encoder-based components for robust control, and suggests applicability to other attributes with minimal parameter overhead.

Abstract

To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 generation, which utilizes a set of small attribute-specific vectors, called prefixes, to steer natural language generation. Different from prefix-tuning, where each prefix is trained independently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single-aspect and multi-aspect control show that our methods can guide generation towards the desired attributes while keeping high linguistic quality.

Figures (5)

• Figure 1: A comparison of prefix-tuning prefix-tuning (top) and our framework (bottom) on sentiment control. The solid arrows show the training process, while the dashed ones show the inference (generation) process. In our proposed framework, the training can be supervised, semi-supervised, or unsupervised.
• Figure 2: An illustration of the GPT2 generation process unfolded through time, controlled by a positive sentiment prefix $H_1=H_{\theta}[1,:,:]$. "The book" is the given prompt. "is good" is the generated completion.
• Figure 3: An illustration of the supervised training method on sentiment control. $H_0$ is the prefix of negative sentiment. $H_1$ is the prefix of positive sentiment. Note that training without $\mathcal{L}_d$ is equivalent to prefix-tuning, where $H_0$ and $H_1$ are trained separately. The GPT2 is pretrained, and its parameters are frozen.
• Figure 4: An illustration of the unsupervised training method. $H_\theta$ denotes the 2 prefixes. $z$ is the latent variable indicating the index of the prefix corresponding to the input text $x$. $\bar{z}$ is the latent variable indicating the index of the opposite prefix. $\otimes$ is matrix multiplication. $\mathcal{L}_{KL}$ is not shown in this figure for clarity.
• Figure 5: The user interface provided to Mechanical Turk workers.