Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Challenges in Restructuring Community-based Moderation

Chau Tran, Kejsi Take, Kaylea Champion, Benjamin Mako Hill, Rachel Greenstadt

TL;DR

The paper investigates why restructuring community-based moderation is difficult in polycentric, self-governed platforms. It combines extensive archival analysis of Flagged Revisions discussions with seven semi-structured interviews to identify eight core challenges across three dimensions: community norms, platform/policy structures, and technical design. The study highlights misalignments between the system's quantitative performance and social norms, governance processes, and workload implications, and proposes a framework for technological change in decentralized settings. The findings offer practical guidance for designing and deploying moderation tools on Wikipedia-like platforms and other distributed communities, emphasizing bottom-up experimentation and cross-stakeholder frame alignment.

Abstract

Content moderation practices and technologies need to change over time as requirements and community expectations shift. However, attempts to restructure existing moderation practices can be difficult, especially for platforms that rely on their communities to conduct moderation activities, because changes can transform the workflow and workload of moderators and contributors' reward systems. Through the study of extensive archival discussions around a prepublication moderation technology on Wikipedia named Flagged Revisions, complemented by seven semi-structured interviews, we identify various challenges in restructuring community-based moderation practices. We learn that while a new system might sound good in theory and perform well in terms of quantitative metrics, it may conflict with existing social norms. Our findings also highlight how the intricate relationship between platforms and self-governed communities can hinder the ability to assess the performance of any new system and introduce considerable costs related to maintaining, overhauling, or scrapping any piece of infrastructure.

Challenges in Restructuring Community-based Moderation

TL;DR

The paper investigates why restructuring community-based moderation is difficult in polycentric, self-governed platforms. It combines extensive archival analysis of Flagged Revisions discussions with seven semi-structured interviews to identify eight core challenges across three dimensions: community norms, platform/policy structures, and technical design. The study highlights misalignments between the system's quantitative performance and social norms, governance processes, and workload implications, and proposes a framework for technological change in decentralized settings. The findings offer practical guidance for designing and deploying moderation tools on Wikipedia-like platforms and other distributed communities, emphasizing bottom-up experimentation and cross-stakeholder frame alignment.

Abstract

Content moderation practices and technologies need to change over time as requirements and community expectations shift. However, attempts to restructure existing moderation practices can be difficult, especially for platforms that rely on their communities to conduct moderation activities, because changes can transform the workflow and workload of moderators and contributors' reward systems. Through the study of extensive archival discussions around a prepublication moderation technology on Wikipedia named Flagged Revisions, complemented by seven semi-structured interviews, we identify various challenges in restructuring community-based moderation practices. We learn that while a new system might sound good in theory and perform well in terms of quantitative metrics, it may conflict with existing social norms. Our findings also highlight how the intricate relationship between platforms and self-governed communities can hinder the ability to assess the performance of any new system and introduce considerable costs related to maintaining, overhauling, or scrapping any piece of infrastructure.
Paper Structure (26 sections, 4 figures, 3 tables)

This paper contains 26 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Peer production and Wikipedia
  4. Community-based moderation
  5. Technological change in organizations
  6. Empirical Setting
  7. Methods
  8. Qualitative Forensic Analysis
  9. Semi-structured Interview Study
  10. Qualitative Data Analysis
  11. Findings
  12. Community challenges
  13. Conflicts with existing social norms and values
  14. Unclear Moderation Instructions.
  15. Platform and Policies Challenges
  16. ...and 11 more sections

Figures (4)

  • Figure 1: An example of reviewing an edit using FlaggedRevs in German Wikipedia
  • Figure 2: Illustration of the challenges between different stakeholders. Most of the platform challenges arise between the Wikimedia Foundation and the different language editions. Most community challenges arise inside the different language editions, among the user base. Technological challenges arise both between the latter and former stakeholders.
  • Figure 3: List of discussions and polls regarding the implementation of Pending Changes on English Wikipedia.
  • Figure 4: Page protection levels and corresponding user accessibility on English Wikipedia.