BIPro: Zero-shot Chinese Poem Generation via Block Inverse Prompting Constrained Generation Framework

TL;DR

Block Inverse Prompting (BIPro) constrained generation framework significantly improves the zero-shot generation quality on the formidable constrained generation task of open-domain traditional-form Chinese poem generation and considerably narrows the gap between AI-generated works and short-listed human literary arts in another human evaluation.

Abstract

Recently, generative pre-trained models have made significant strides, particularly highlighted by the release of ChatGPT and GPT-4, which exhibit superior cross-domain capabilities. However, these models still face challenges on constrained writing tasks like poem generation under open-domain titles. In response to this challenge, we introduce Block Inverse Prompting (BIPro) constrained generation framework. BIPro leverages two block inverse prompting methods, revise and rewrite, that mimic the process of human text writing using block generative models. It significantly improves the zero-shot generation quality on the formidable constrained generation task of open-domain traditional-form Chinese poem generation. Based on a less powerful block generative model GLM-10B-Chinese, poems composed via BIPro without priming or additional training outperform both most advanced direct generative systems like GPT-4 or GLM-4 and best domain-specific systems such as Yusheng, Shisanbai, or Baidu Poetry Helper in human evaluation by proficient poets. Finally, BIPro considerably narrows the gap between AI-generated works and short-listed human literary arts in another human evaluation, unveiling the promising potential of block generative models in improving the quality of constrained generation.

Figures (8)

• Figure 1: The generation process of poem "Lament over Life" under BIPro framework. Sentences are generated with constraints using block generative model. Each sentence is revised after its subsequent sentence is generated. The full poem endures several rounds of rewrite.
• Figure 2: Examples of formats used in direct inverse prompting and BIPro. BIPro directly masks the prompt and evaluate the perplexity under block generative models, skipping the inverse transformation process in direct inverse prompting.
• Figure 3: Beam-based constrained generation. Bad generation are replaced by good generations from other beams at each step. Finally, generations that satisfy constraints are scored and selected accordingly.
• Figure 4: BIPro scorer. The input is first transformed to BIPro prompt and target text, then BIPro prompt is fed into block generative model and the perplexity of the target text is used for scoring.
• Figure 5: A representative case in parallel poem generation challenge.