Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Change My Frame: Reframing in the Wild in r/ChangeMyView

Arturo Martínez Peguero, Taro Watanabe

TL;DR

The paper tackles reframing beyond positive framing by harvesting authentic, community-recognized reframing signals from r/ChangeMyView. It builds a dataset of 32,306 (post,comment) pairs where the OP awarded a delta to a direct reply, leveraging the DeltaBot mechanism. Using BART and T5, the authors fine-tune on this data and explore LLM-assisted data curation with GPT-4, finding that substantial data trimming improves semantic similarity metrics but cross-dataset transfer remains challenging. They conclude that data condensation and richer evaluation, including linguistic-attribute frameworks, are needed and that extending to other languages can broaden applicability.

Abstract

Recent work in reframing, within the scope of text style transfer, has so far made use of out-of-context, task-prompted utterances in order to produce neutralizing or optimistic reframes. Our work aims to generalize reframing based on the subreddit r/ChangeMyView (CMV). We build a dataset that leverages CMV's community's interactions and conventions to identify high-value, community-recognized utterances that produce changes of perspective. With this data, we widen the scope of the direction of reframing since the changes in perspective do not only occur in neutral or positive directions. We fine tune transformer-based models, make use of a modern LLM to refine our dataset, and explore challenges in the dataset creation and evaluation around this type of reframing.

Change My Frame: Reframing in the Wild in r/ChangeMyView

TL;DR

The paper tackles reframing beyond positive framing by harvesting authentic, community-recognized reframing signals from r/ChangeMyView. It builds a dataset of 32,306 (post,comment) pairs where the OP awarded a delta to a direct reply, leveraging the DeltaBot mechanism. Using BART and T5, the authors fine-tune on this data and explore LLM-assisted data curation with GPT-4, finding that substantial data trimming improves semantic similarity metrics but cross-dataset transfer remains challenging. They conclude that data condensation and richer evaluation, including linguistic-attribute frameworks, are needed and that extending to other languages can broaden applicability.

Abstract

Recent work in reframing, within the scope of text style transfer, has so far made use of out-of-context, task-prompted utterances in order to produce neutralizing or optimistic reframes. Our work aims to generalize reframing based on the subreddit r/ChangeMyView (CMV). We build a dataset that leverages CMV's community's interactions and conventions to identify high-value, community-recognized utterances that produce changes of perspective. With this data, we widen the scope of the direction of reframing since the changes in perspective do not only occur in neutral or positive directions. We fine tune transformer-based models, make use of a modern LLM to refine our dataset, and explore challenges in the dataset creation and evaluation around this type of reframing.
Paper Structure (10 sections, 1 figure, 3 tables)

This paper contains 10 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Previous Work
  3. Dataset Building
  4. r/ChangeMyView
  5. Leveraging the delta system
  6. r/ChangeMyView reframing data
  7. Experiments & Evaluation
  8. Results & Discussion
  9. Conclusion and future work
  10. Appendix A: Results from more experiments

Figures (1)

  • Figure 1: Data collection mechanism described in section \ref{['subsec:leveragingdelta']}. We look for posts' direct replies that change views and that are recognized as such by the original post (OP) author.