Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Investigating Societal Biases in a Poetry Composition System

Emily Sheng, David Uthus

TL;DR

This work addresses societal biases in a creative NLP task—retrieving next-verse suggestions in a poetry composition system. It introduces a bias-mitigation pipeline based on sentiment-style transfer, including a labeled poetry sentiment dataset and a BERT sentiment classifier to drive data augmentation via the DRG approach. By augmenting the next-verse training data with sentiment-shifted verses, the dual-encoder retrieval model is encouraged to retrieve less negative, more positive verses while preserving content. Results show modest yet consistent improvements in sentiment of retrieved verses with comparable relevance and usability, demonstrating the potential of style-transfer augmentation to reduce bias in retrieval for creative language tasks.

Abstract

There is a growing collection of work analyzing and mitigating societal biases in language understanding, generation, and retrieval tasks, though examining biases in creative tasks remains underexplored. Creative language applications are meant for direct interaction with users, so it is important to quantify and mitigate societal biases in these applications. We introduce a novel study on a pipeline to mitigate societal biases when retrieving next verse suggestions in a poetry composition system. Our results suggest that data augmentation through sentiment style transfer has potential for mitigating societal biases.

Investigating Societal Biases in a Poetry Composition System

TL;DR

This work addresses societal biases in a creative NLP task—retrieving next-verse suggestions in a poetry composition system. It introduces a bias-mitigation pipeline based on sentiment-style transfer, including a labeled poetry sentiment dataset and a BERT sentiment classifier to drive data augmentation via the DRG approach. By augmenting the next-verse training data with sentiment-shifted verses, the dual-encoder retrieval model is encouraged to retrieve less negative, more positive verses while preserving content. Results show modest yet consistent improvements in sentiment of retrieved verses with comparable relevance and usability, demonstrating the potential of style-transfer augmentation to reduce bias in retrieval for creative language tasks.

Abstract

There is a growing collection of work analyzing and mitigating societal biases in language understanding, generation, and retrieval tasks, though examining biases in creative tasks remains underexplored. Creative language applications are meant for direct interaction with users, so it is important to quantify and mitigate societal biases in these applications. We introduce a novel study on a pipeline to mitigate societal biases when retrieving next verse suggestions in a poetry composition system. Our results suggest that data augmentation through sentiment style transfer has potential for mitigating societal biases.

Paper Structure

This paper contains 34 sections, 4 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Poetry Composition System
  3. Next verse prediction loss.
  4. Next verse prediction dataset.
  5. Bias Mitigation Through Data Augmentation
  6. Definitions
  7. Sentiment Analysis
  8. Dataset.
  9. Sentiment classifier.
  10. Style Transfer
  11. Delete, Retrieve, Generate (DRG) technique.
  12. Experimental setup.
  13. Human evaluation.
  14. Next Verse Prediction
  15. Data augmentation.
  16. ...and 19 more sections

Figures (3)

  • Figure 1: An overview of the poetry composition system, focusing on the components that make up the next verse prediction.
  • Figure 2: A schematic of our technique for bias mitigation of the next verse prediction component in the poetry composition system. Solid lines indicate training time; dashed lines indicate inference time.
  • Figure 3: Data augmentation details. For example, input verse = "by the path an indian sat", next verse = "then i cried and ran away", and the positive sentiment style-transferred next verse ST next verse = "then i sing that human delight". (input verse, next verse) is the groundtruth data pair in the Gutenberg Poem Dataset. In this example, input verse contains a demographic mention ("indian") and next verse has negative sentiment. The baseline next verse prediction model uses the original (input verse, next verse) pair as a positive example. The data augmentation model treats (input verse, next verse) as a negative example and uses (input verse, ST next verse) as the positive example.