Generative Concatenative Nets Jointly Learn to Write and Classify Reviews

TL;DR

The paper introduces the Generative Concatenative Network (GCN), a character-level LSTM conditioned on auxiliary inputs to generate personalized beer reviews and simultaneously perform classification tasks. By replicating auxiliary signals at each time step and employing weight transplantation, the GCN achieves significantly lower perplexities than standard language models and competitive or superior results for author identification, category prediction, and sentiment classification, all while capturing nonlinear textual phenomena such as negation. The work demonstrates both high-quality text generation and useful predictive capabilities without explicit supervision for classification, suggesting practical applications in richer, more personalized recommender systems. It also discusses limitations in inference speed for large class counts and outlines directions for scaling and improving efficiency.

Abstract

A recommender system's basic task is to estimate how users will respond to unseen items. This is typically modeled in terms of how a user might rate a product, but here we aim to extend such approaches to model how a user would write about the product. To do so, we design a character-level Recurrent Neural Network (RNN) that generates personalized product reviews. The network convincingly learns styles and opinions of nearly 1000 distinct authors, using a large corpus of reviews from BeerAdvocate.com. It also tailors reviews to describe specific items, categories, and star ratings. Using a simple input replication strategy, the Generative Concatenative Network (GCN) preserves the signal of static auxiliary inputs across wide sequence intervals. Without any additional training, the generative model can classify reviews, identifying the author of the review, the product category, and the sentiment (rating), with remarkable accuracy. Our evaluation shows the GCN captures complex dynamics in text, such as the effect of negation, misspellings, slang, and large vocabularies gracefully absent any machinery explicitly dedicated to the purpose.

Figures (6)

• Figure 1: Real and synthetic reviews by user Mosstrooper describing the beer Shock Top Belgian White. Note that this user and item combination never occurs in the training set. The GCN captures Mosstrooper's idiosyncracies, including his/her tendency to mark each aspect of the beer with fully spelled-out headings (Appearance, Smell, etc., in a particular order). The GCN also accurately predicts that Mosstrooper will comment on the beer's golden color, citrus flavor, and hints of wheat (highlighted in color).
• Figure 2: Our generative model infers ratings and categories given reviews without any a priori notion of words.
• Figure 3: (a) Standard generative RNN; (b) encoder-decoder RNN; (c) concatenated input RNN.
• Figure 4: Probability of each category as each character in the review is encountered. The GCN learns Budweiser is a lager and that stouts and porters are heavy.
• Figure 5: Most likely star rating as each letter is encountered. The GCN learns to tilt positive by the 'c' in 'excellent' and that the 'f' in 'awful' reveals negative sentiment.