Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Controllable Text Generation with Residual Memory Transformer

Hanqing Zhang, Sun Si, Haiming Wu, Dawei Song

TL;DR

The paper tackles controllable text generation (CTG) for large causal language models by introducing a non-intrusive, lightweight control plug-in called Residual Memory Transformer (RMT). RMT uses an encoder–decoder architecture to encode control signals and fuse them with a frozen CLM's generation through residual learning, enabling control at arbitrary steps with minimal overhead. Key contributions include a detailed RMT design with three targeted attention streams, a denoising auto-encoder pre-training regime, and task-specific fine-tuning that achieves competitive or superior control on word inclusion, length control, and sentiment, while preserving text quality and generation speed. The approach offers a flexible, parameter-efficient path to augment existing CLMs without full model fine-tuning, with potential for extension to retrieval-augmented or multimodal generation.

Abstract

Large-scale Causal Language Models (CLMs), e.g., GPT3 and ChatGPT, have brought great success in text generation. However, it is still an open challenge to control the generation process of CLM while balancing flexibility, control granularity, and generation efficiency. In this paper, we provide a new alternative for controllable text generation (CTG), by designing a non-intrusive, lightweight control plugin to accompany the generation of CLM at arbitrary time steps. The proposed control plugin, namely Residual Memory Transformer (RMT), has an encoder-decoder setup, which can accept any types of control conditions and cooperate with CLM through a residual learning paradigm, to achieve a more flexible, general, and efficient CTG. Extensive experiments are carried out on various control tasks, in the form of both automatic and human evaluations. The results show the superiority of RMT over a range of state-of-the-art approaches, proving the effectiveness and versatility of our approach.

Controllable Text Generation with Residual Memory Transformer

TL;DR

The paper tackles controllable text generation (CTG) for large causal language models by introducing a non-intrusive, lightweight control plug-in called Residual Memory Transformer (RMT). RMT uses an encoder–decoder architecture to encode control signals and fuse them with a frozen CLM's generation through residual learning, enabling control at arbitrary steps with minimal overhead. Key contributions include a detailed RMT design with three targeted attention streams, a denoising auto-encoder pre-training regime, and task-specific fine-tuning that achieves competitive or superior control on word inclusion, length control, and sentiment, while preserving text quality and generation speed. The approach offers a flexible, parameter-efficient path to augment existing CLMs without full model fine-tuning, with potential for extension to retrieval-augmented or multimodal generation.

Abstract

Large-scale Causal Language Models (CLMs), e.g., GPT3 and ChatGPT, have brought great success in text generation. However, it is still an open challenge to control the generation process of CLM while balancing flexibility, control granularity, and generation efficiency. In this paper, we provide a new alternative for controllable text generation (CTG), by designing a non-intrusive, lightweight control plugin to accompany the generation of CLM at arbitrary time steps. The proposed control plugin, namely Residual Memory Transformer (RMT), has an encoder-decoder setup, which can accept any types of control conditions and cooperate with CLM through a residual learning paradigm, to achieve a more flexible, general, and efficient CTG. Extensive experiments are carried out on various control tasks, in the form of both automatic and human evaluations. The results show the superiority of RMT over a range of state-of-the-art approaches, proving the effectiveness and versatility of our approach.
Paper Structure (25 sections, 8 equations, 3 figures, 11 tables)

This paper contains 25 sections, 8 equations, 3 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Attention in Transformer
  4. Causal Language Model
  5. Methodology
  6. CTG Framework with RMT
  7. Residual Memory Transformer (RMT)
  8. Model Training
  9. Experiments
  10. Word Inclusion Experiment
  11. Sentence Length Control Experiment
  12. Attribute Control Experiment
  13. Further Analysis
  14. Related Work
  15. Discussion and Future Work
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Illustration of controllable text generation with Residual Memory Transformer (RMT). In the top left, we present a miniaturization of our method framework, where the gray sector represents the frozen CLM, and the red box symbolizes our plug-in control module, i.e., RMT. The right part illustrates the embodiment of each detailed component.
  • Figure 2: Control length and control performance.
  • Figure 3: RMT block layers and control performance. RMT could achieve a consistently high and stable control performance for CLMs with different sizes. About three block layers are sufficient for optimal performance.