Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer

Zachary Horvitz, Ajay Patel, Chris Callison-Burch, Zhou Yu, Kathleen McKeown

TL;DR

ParaGuide introduces a diffusion-based framework for unsupervised textual style transfer that can adapt to arbitrary target styles without retraining. It uses paraphrase-conditioned diffusion with gradient guidance from off-the-shelf attribute classifiers and authorship style embedders to steer outputs while preserving meaning. Across Enron-based evaluations, ParaGuide achieves strong attribute transfer performance, competitive human-formal evaluation results, and notable gains in low-resource authorship transfer, with controllable trade-offs via a guidance strength parameter. This work shows diffusion models as a viable, plug-and-play alternative to autoregressive methods for controllable text generation and style transfer.

Abstract

Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. Target "styles" can be defined in numerous ways, ranging from single attributes (e.g, formality) to authorship (e.g, Shakespeare). Previous unsupervised style-transfer approaches generally rely on significant amounts of labeled data for only a fixed set of styles or require large language models. In contrast, we introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles at inference time. Our parameter-efficient approach, ParaGuide, leverages paraphrase-conditioned diffusion models alongside gradient-based guidance from both off-the-shelf classifiers and strong existing style embedders to transform the style of text while preserving semantic information. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.

ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer

TL;DR

ParaGuide introduces a diffusion-based framework for unsupervised textual style transfer that can adapt to arbitrary target styles without retraining. It uses paraphrase-conditioned diffusion with gradient guidance from off-the-shelf attribute classifiers and authorship style embedders to steer outputs while preserving meaning. Across Enron-based evaluations, ParaGuide achieves strong attribute transfer performance, competitive human-formal evaluation results, and notable gains in low-resource authorship transfer, with controllable trade-offs via a guidance strength parameter. This work shows diffusion models as a viable, plug-and-play alternative to autoregressive methods for controllable text generation and style transfer.

Abstract

Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. Target "styles" can be defined in numerous ways, ranging from single attributes (e.g, formality) to authorship (e.g, Shakespeare). Previous unsupervised style-transfer approaches generally rely on significant amounts of labeled data for only a fixed set of styles or require large language models. In contrast, we introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles at inference time. Our parameter-efficient approach, ParaGuide, leverages paraphrase-conditioned diffusion models alongside gradient-based guidance from both off-the-shelf classifiers and strong existing style embedders to transform the style of text while preserving semantic information. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
Paper Structure (52 sections, 14 equations, 2 figures, 10 tables, 1 algorithm)

This paper contains 52 sections, 14 equations, 2 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. ParaGuide
  4. Overview
  5. Initial Paraphrase Generation
  6. Paraphrase-Conditioned Diffusion
  7. Diffusion
  8. Categorical Reparameterization
  9. Diffusion Model Architecture
  10. Diffusion Model Loss
  11. Diffusion Noise Schedule
  12. Diffusion Model Inference
  13. Guidance
  14. Attribute Classifiers
  15. Style Embedding Distance
  16. ...and 37 more sections

Figures (2)

  • Figure 1: We train paraphrase-conditioned text diffusion models to reconstruct semantically consistent text from noised word embeddings. At inference time, we guide the reconstruction towards target styles with off-the-shelf models.
  • Figure 2: As we increase the guidance hyperparameter $\lambda$, we steadily increase style transfer accuracy (Confusion), at the cost of semantic consistency (Sim) and Fluency.