Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor

Sangwon Yu, Changmin Lee, Hojin Lee, Sungroh Yoon

TL;DR

ScoPE tackles the challenge of controllable text generation under black-box language models by editing generated text in token blocks to align with target attributes without requiring parameter access. It introduces a score-guided, block-wise editor that maximizes a target score while preserving fluency, supported by offline training, fine-tuning a masked language model on the target domain, and a token-level score-disparity objective. The paper provides detailed formulations of a target MLM score, a repetition penalty, and a task-specific score, and demonstrates the method across category, sentiment, and multi-attribute control tasks using Amazon reviews and multiple backbones. The findings indicate ScoPE can robustly steer generation toward desired attributes while maintaining high fluency, with practical implications for adapting large language models to domain-specific needs in real-world settings, and it includes code for reproducibility.

Abstract

Controlled text generation is very important for the practical use of language models because it ensures that the produced text includes only the desired attributes from a specific domain or dataset. Existing methods, however, are inapplicable to black-box models or suffer a significant trade-off between controlling the generated text and maintaining its fluency. This paper introduces the Score-based Progressive Editor (ScoPE), a novel approach designed to overcome these issues. ScoPE modifies the context at the token level during the generation process of a backbone language model. This modification guides the subsequent text to naturally include the target attributes. To facilitate this process, ScoPE employs a training objective that maximizes a target score, thoroughly considering both the ability to guide the text and its fluency. Experimental results on diverse controlled generation tasks demonstrate that ScoPE can effectively regulate the attributes of the generated text while fully utilizing the capability of the backbone large language models. Our codes are available at \url{https://github.com/ysw1021/ScoPE}.

Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor

TL;DR

ScoPE tackles the challenge of controllable text generation under black-box language models by editing generated text in token blocks to align with target attributes without requiring parameter access. It introduces a score-guided, block-wise editor that maximizes a target score while preserving fluency, supported by offline training, fine-tuning a masked language model on the target domain, and a token-level score-disparity objective. The paper provides detailed formulations of a target MLM score, a repetition penalty, and a task-specific score, and demonstrates the method across category, sentiment, and multi-attribute control tasks using Amazon reviews and multiple backbones. The findings indicate ScoPE can robustly steer generation toward desired attributes while maintaining high fluency, with practical implications for adapting large language models to domain-specific needs in real-world settings, and it includes code for reproducibility.

Abstract

Controlled text generation is very important for the practical use of language models because it ensures that the produced text includes only the desired attributes from a specific domain or dataset. Existing methods, however, are inapplicable to black-box models or suffer a significant trade-off between controlling the generated text and maintaining its fluency. This paper introduces the Score-based Progressive Editor (ScoPE), a novel approach designed to overcome these issues. ScoPE modifies the context at the token level during the generation process of a backbone language model. This modification guides the subsequent text to naturally include the target attributes. To facilitate this process, ScoPE employs a training objective that maximizes a target score, thoroughly considering both the ability to guide the text and its fluency. Experimental results on diverse controlled generation tasks demonstrate that ScoPE can effectively regulate the attributes of the generated text while fully utilizing the capability of the backbone large language models. Our codes are available at \url{https://github.com/ysw1021/ScoPE}.
Paper Structure (42 sections, 10 equations, 2 figures, 26 tables, 3 algorithms)

This paper contains 42 sections, 10 equations, 2 figures, 26 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. ScoPE: Score-based Progressive Editor
  4. Approach for Controlled Text Generation
  5. Overview
  6. Block-Wise Editing
  7. Training ScoPE
  8. Preparation for ScoPE Training
  9. Training set for offline learning framework
  10. Fine-tuning pre-trained MLM
  11. Maximizing Score Disparity between Input and Edited Texts
  12. Techniques for stable training
  13. Score Formulation for Target Attributes
  14. Target Score from Fine-tuned MLM
  15. Repetition Score
  16. ...and 27 more sections

Figures (2)

  • Figure 1: Overview of the controlled text generation for black-box LM with ScoPE. Starting from the neutral initial input sequence $\mathbf{x}$, ScoPE edits every $b$ tokens, $\tilde{\mathbf{y}}$, generated from $p_{LM}(\cdot | \mathbf{x})$ to $\hat{\mathbf{y}}$. $\hat{\mathbf{y}}$ has enhanced target score compared to $\tilde{\mathbf{y}}$, which means $\hat{\mathbf{y}}$ is closer to the target distribution. Edited subtext $(\mathbf{x}, \hat{\mathbf{y}})$ become a new input for the next generation step of black-box LM, guiding subsequent generations to contain the target attributes.
  • Figure 2: Overview of the mechanism of the two approaches for the multi-attributes controlled generation using ScoPE: mixed logit (ML), sequential editing (SE).