Controllable Generation from Pre-trained Language Models via Inverse Prompting

TL;DR

Inverse prompting leverages the original language model to score generated text by the inverse likelihood of recovering the prompt, integrated into beam search to improve controllability. By transforming prompts and context into inverse formats, the method strengthens the alignment between prompts and outputs without extra attribute models. Across open-domain long-form Chinese QA and traditional Chinese poem generation, inverse prompting yields substantial gains over prompting baselines and competitive performance vs. state-of-the-art systems, approaching human quality on several tasks. Self-training further enhances poem generation, indicating the method's strong potential for broad controllable generation with existing models and data.

Abstract

Large-scale pre-trained language models have demonstrated strong capabilities of generating realistic text. However, it remains challenging to control the generation results. Previous approaches such as prompting are far from sufficient, which limits the usage of language models. To tackle this challenge, we propose an innovative method, inverse prompting, to better control text generation. The core idea of inverse prompting is to use generated text to inversely predict the prompt during beam search, which enhances the relevance between the prompt and the generated text and provides better controllability. Empirically, we pre-train a large-scale Chinese language model to perform a systematic study using human evaluation on the tasks of open-domain poem generation and open-domain long-form question answering. Our results show that our proposed method substantially outperforms the baselines and that our generation quality is close to human performance on some of the tasks. Narrators can try our poem generation demo at https://pretrain.aminer.cn/apps/poetry.html, while our QA demo can be found at https://pretrain.aminer.cn/app/qa. For researchers, the code is provided in https://github.com/THUDM/InversePrompting.

Figures (11)

• Figure 1: The generation process of open-domain traditional Chinese poems under inverse prompting. Using title New York as an example.
• Figure 2: An example showing how the prompting baseline model may fail to maintain relevance in generated text, and how inverse prompting alleiates this issue. The relevance and overall scores were obtained from human evaluation.
• Figure 3: Language model generation and language model inverse prompting scoring for generating a poem sentence.
• Figure 4: Inverse prompting transformation Table. The first rows represents the inverse prompts used in experiments.(in Chinese and English) Some additional examples of inverse prompting format are also displayed.
• Figure 5: A Perfect Example of inverse prompting generating better answer than human in open-domain long-form QA.