Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

TL;DR

The paper tackles unsupervised text attribute transfer by identifying and deleting attribute-marking phrases to isolate content, then retrieving target-attribute cues and generating fluent outputs. It formalizes four methods—RetrieveOnly, TemplateBased, DeleteOnly, and DeleteAndRetrieve—along with a denoising auto-encoder training regime to avoid trivial reconstructions. Across Yelp, Amazon, and Captions benchmarks, the DeleteAndRetrieve approach yields the highest human-evaluated success, significantly outperforming prior adversarial methods and enabling controllable trade-offs between content preservation and attribute change. The work demonstrates a practical, scalable alternative to adversarial training with publicly available code and data for reproducibility.

Abstract

We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., changing "screen is just the right size" to "screen is too small"). Our training data includes only sentences labeled with their attribute (e.g., positive or negative), but not pairs of sentences that differ only in their attributes, so we must learn to disentangle attributes from attribute-independent content in an unsupervised way. Previous work using adversarial methods has struggled to produce high-quality outputs. In this paper, we propose simpler methods motivated by the observation that text attributes are often marked by distinctive phrases (e.g., "too small"). Our strongest method extracts content words by deleting phrases associated with the sentence's original attribute value, retrieves new phrases associated with the target attribute, and uses a neural model to fluently combine these into a final output. On human evaluation, our best method generates grammatical and appropriate responses on 22% more inputs than the best previous system, averaged over three attribute transfer datasets: altering sentiment of reviews on Yelp, altering sentiment of reviews on Amazon, and altering image captions to be more romantic or humorous.

Figures (4)

• Figure 1: An overview of our approach. (a) We identify attribute markers from an unaligned corpus. (b) We transfer attributes by removing markers of the original attribute, then generating a new sentence conditioned on the remaining words and the target attribute.
• Figure 2: Our four proposed methods on the same sentence, taken from the Amazon dataset. Every method uses the same procedure (1) to separate attribute and content by deleting attribute markers; they differ in the construction of the target sentence. RetrieveOnly directly returns the sentence retrieved in (2). TemplateBased combines the content with the target attribute markers in the retrieved sentence by slot filling. DeleteAndRetrieve generates the output from the content and the retrieved target attribute markers with an RNN. DeleteOnly generates the output from the content and the target attribute with an RNN.
• Figure 3: Trade-off curves between matching the target attribute (measured by classifier scores) and preserving the content (measured by BLEU). Bigger points on the curve correspond to settings used for both training and our official evaluation.
• Figure 4: Scatter plots of humans scores vs. automatic metric scores on attribute transfer, content preservation, and grammaticality. The automatic metrics have some correlation with the attribute transfer and content preservation ratings, though results vary across datasets; the metrics do not measure grammaticality.