Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community

Giuseppe Russo, Maciej Styczen, Manoel Horta Ribeiro, Robert West

TL;DR

The paper examines whether platform sanctions on fringe Manosphere communities drive users toward the exredpill recovery community. It applies ITS and BSTS causal inference to measure the effects of quarantines and bans on exredpill, and contrasts these with effects from related real-world events. Results show hard moderation (bans) substantially increases exredpill activity, newcomer participation, and cross-community migration, while soft moderation (quarantines) yields little to no robust impact; real-world events have smaller effects. The authors conclude that moderation can act as a deradicalization catalyst and discuss practical and ethical implications for platform policy and future research.

Abstract

Online platforms have sanctioned individuals and communities associated with fringe movements linked to hate speech, violence, and terrorism, but can these sanctions contribute to the abandonment of these movements? Here, we investigate this question through the lens of exredpill, a recovery community on Reddit meant to help individuals leave movements within the Manosphere, a conglomerate of fringe Web based movements focused on men's issues. We conduct an observational study on the impact of sanctioning some of Reddit's largest Manosphere communities on the activity levels and user influx of exredpill, the largest associated recovery subreddit. We find that banning a related radical community positively affects participation in exredpill in the period following the ban. Yet, quarantining the community, a softer moderation intervention, yields no such effects. We show that the effect induced by banning a radical community is stronger than for some of the widely discussed real-world events related to the Manosphere and that moderation actions against the Manosphere do not cause a spike in toxicity or malicious activity in exredpill. Overall, our findings suggest that content moderation acts as a deradicalization catalyst.

Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community

TL;DR

The paper examines whether platform sanctions on fringe Manosphere communities drive users toward the exredpill recovery community. It applies ITS and BSTS causal inference to measure the effects of quarantines and bans on exredpill, and contrasts these with effects from related real-world events. Results show hard moderation (bans) substantially increases exredpill activity, newcomer participation, and cross-community migration, while soft moderation (quarantines) yields little to no robust impact; real-world events have smaller effects. The authors conclude that moderation can act as a deradicalization catalyst and discuss practical and ethical implications for platform policy and future research.

Abstract

Online platforms have sanctioned individuals and communities associated with fringe movements linked to hate speech, violence, and terrorism, but can these sanctions contribute to the abandonment of these movements? Here, we investigate this question through the lens of exredpill, a recovery community on Reddit meant to help individuals leave movements within the Manosphere, a conglomerate of fringe Web based movements focused on men's issues. We conduct an observational study on the impact of sanctioning some of Reddit's largest Manosphere communities on the activity levels and user influx of exredpill, the largest associated recovery subreddit. We find that banning a related radical community positively affects participation in exredpill in the period following the ban. Yet, quarantining the community, a softer moderation intervention, yields no such effects. We show that the effect induced by banning a radical community is stronger than for some of the widely discussed real-world events related to the Manosphere and that moderation actions against the Manosphere do not cause a spike in toxicity or malicious activity in exredpill. Overall, our findings suggest that content moderation acts as a deradicalization catalyst.
Paper Structure (11 sections, 1 equation, 3 figures, 8 tables)

This paper contains 11 sections, 1 equation, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Materials and Methods
  4. Data Collection
  5. Estimating the Causal Effect
  6. Results
  7. RQ1: Effect of quarantining
  8. RQ2: Effect of banning
  9. RQ3: Comparison with Real-World Events
  10. Robustness Checks
  11. Discussion

Figures (3)

  • Figure 1: Some of the most upvoted comments and submissions in the r/exredpill recovery community (paraphrased due to privacy concerns).
  • Figure 2: Effects of Quarantine on r/exredpill Activity Volume, Newcomers, and Migrants. We show the results obtained after fitting the ITS model on the activity volume, newcomers, and migrants that joined r/exredpill in the aftermath of the quarantine. In the left column (a), we show the ITS analysis assessing the effect on activity volume and newcomers after the quarantine of r/Braincels and r/TheRedPill. In right column (b), we show the same analysis after the quarantine of r/MGTOW. In the bottom row, we show the ITS analysis for the users who joined r/exredpill after having participated in r/Braincels (a), r/TheRedPill (b), and r/MGTOW (c).
  • Figure 3: Effects of Banning on r/exredpill Activity Volume, Newcomers, and Migrants. We show the results obtained after fitting the ITS model on the activity volume, newcomers, and migrants that joined r/exredpill in the aftermath of the banning. In the left column (a), we show the ITS analysis assessing the effect on activity volume and newcomers after the ban of r/Braincels. In right column (b), we show the same analysis after the banning of r/MGTOW.