Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AppealMod: Inducing Friction to Reduce Moderator Workload of Handling User Appeals

Shubham Atreja, Jane Im, Paul Resnick, Libby Hemphill

TL;DR

This paper investigates how volunteer moderators shoulder heavy workloads in handling user ban appeals and proposes AppealMod, a friction-based system that requires appealing users to provide additional information before human review. Through a collaborative, iterative design process with Reddit moderators and a four-month field experiment in r/pics, AppealMod uses an automated bot to guide users through a structured webform, hides incomplete appeals, and surfaces rich user responses only after completion. The study finds that AppealMod reduces moderator workload by making 70% of appeals invisible to moderators, lowers exposure to toxic content, and does not reduce the overall number of appeals granted; some users with higher initial likelihood of success are more likely to complete the process, indicating a selective effect. The work demonstrates that carefully designed friction can streamline moderation without sacrificing direct human engagement, with implications for scalable contestability of decisions and potential adaptation to other communities and platforms.

Abstract

As content moderation becomes a central aspect of all social media platforms and online communities, interest has grown in how to make moderation decisions contestable. On social media platforms where individual communities moderate their own activities, the responsibility to address user appeals falls on volunteers from within the community. While there is a growing body of work devoted to understanding and supporting the volunteer moderators' workload, little is known about their practice of handling user appeals. Through a collaborative and iterative design process with Reddit moderators, we found that moderators spend considerable effort in investigating user ban appeals and desired to directly engage with users and retain their agency over each decision. To fulfill their needs, we designed and built AppealMod, a system that induces friction in the appeals process by asking users to provide additional information before their appeals are reviewed by human moderators. In addition to giving moderators more information, we expected the friction in the appeal process would lead to a selection effect among users, with many insincere and toxic appeals being abandoned before getting any attention from human moderators. To evaluate our system, we conducted a randomized field experiment in a Reddit community of over 29 million users that lasted for four months. As a result of the selection effect, moderators viewed only 30% of initial appeals and less than 10% of the toxically worded appeals; yet they granted roughly the same number of appeals when compared with the control group. Overall, our system is effective at reducing moderator workload and minimizing their exposure to toxic content while honoring their preference for direct engagement and agency in appeals.

AppealMod: Inducing Friction to Reduce Moderator Workload of Handling User Appeals

TL;DR

This paper investigates how volunteer moderators shoulder heavy workloads in handling user ban appeals and proposes AppealMod, a friction-based system that requires appealing users to provide additional information before human review. Through a collaborative, iterative design process with Reddit moderators and a four-month field experiment in r/pics, AppealMod uses an automated bot to guide users through a structured webform, hides incomplete appeals, and surfaces rich user responses only after completion. The study finds that AppealMod reduces moderator workload by making 70% of appeals invisible to moderators, lowers exposure to toxic content, and does not reduce the overall number of appeals granted; some users with higher initial likelihood of success are more likely to complete the process, indicating a selective effect. The work demonstrates that carefully designed friction can streamline moderation without sacrificing direct human engagement, with implications for scalable contestability of decisions and potential adaptation to other communities and platforms.

Abstract

As content moderation becomes a central aspect of all social media platforms and online communities, interest has grown in how to make moderation decisions contestable. On social media platforms where individual communities moderate their own activities, the responsibility to address user appeals falls on volunteers from within the community. While there is a growing body of work devoted to understanding and supporting the volunteer moderators' workload, little is known about their practice of handling user appeals. Through a collaborative and iterative design process with Reddit moderators, we found that moderators spend considerable effort in investigating user ban appeals and desired to directly engage with users and retain their agency over each decision. To fulfill their needs, we designed and built AppealMod, a system that induces friction in the appeals process by asking users to provide additional information before their appeals are reviewed by human moderators. In addition to giving moderators more information, we expected the friction in the appeal process would lead to a selection effect among users, with many insincere and toxic appeals being abandoned before getting any attention from human moderators. To evaluate our system, we conducted a randomized field experiment in a Reddit community of over 29 million users that lasted for four months. As a result of the selection effect, moderators viewed only 30% of initial appeals and less than 10% of the toxically worded appeals; yet they granted roughly the same number of appeals when compared with the control group. Overall, our system is effective at reducing moderator workload and minimizing their exposure to toxic content while honoring their preference for direct engagement and agency in appeals.
Paper Structure (54 sections, 10 figures, 8 tables)

This paper contains 54 sections, 10 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Volunteer Moderator Workload and Experiences
  4. Designing Tools for Volunteer Moderators
  5. Designing for Contestability of Moderation Decisions
  6. Cost and Friction for Reducing Problematic User Behavior in Content Moderation
  7. Study Context: Ban Appeals on Reddit
  8. Design Process and Moderator Needs
  9. Method: Collaborative and Iterative Design Process with Moderators
  10. Building connections with moderators
  11. Understanding moderators' handling of users ban appeals
  12. Iterative design process
  13. Pilot deployment
  14. Findings: Understanding Moderators' Practices and Needs
  15. AppealMod
  16. ...and 39 more sections

Figures (10)

  • Figure 1: An example ban message shared with the banned user.
  • Figure 2: An example interaction between an appealing user and human moderators following the user's ban.
  • Figure 3: Two examples demonstrating a complete (left) and incomplete (right) AppealMod process. A) Our bot responds to the user's appeal and shares a link to the form containing our questions. B) Bot archives the conversation to hide it from human moderators. C) Once the user completes the AppealMod process, the bot hands over their appeal to human moderators. D) If the user sends any more messages before completing the process, they are reminded to complete the process. E) For users who complete the process, a private note summarizing their responses is also shared with human moderators.
  • Figure 4: An example demonstrating how a user's responses are formatted and shared with human moderators (E). The information is readily available to moderators along with the user's past history already provided by Reddit (G). Moderators make the final decision by directly interacting with the user (F).
  • Figure 5: AppealMod Bot Dialogue Flow
  • ...and 5 more figures