Towards a Better Modqueue: Designing for Diversity Across Moderator Objectives and Workflows

TL;DR

This paper investigates what constitutes improvement in Reddit's modqueue by examining the diverse objectives moderators bring to review and their perceptions of lightweight interventions. It combines a survey of 106 moderators with a modular agent-based simulation to audit modqueue practices, explore tradeoffs among objectives such as accuracy, fairness, wellbeing, efficiency, and redundancy, and assess potential interventions like awareness indicators and alternative sorting. The key contributions are (1) empirical identification of moderator objectives, (2) assessment of intervention perceptions, and (3) a simulation-based audit framework that can probe tradeoffs and guide design. The work demonstrates that there is no single objective to optimize and that modular, configurable designs coupled with transparent simulations are valuable for improving moderator workflows while respecting community-specific practices and values.

Abstract

Reddit relies on volunteer moderators to enforce community rules, configure tools, and review flagged content. This labor is substantial, worth millions in unpaid effort, and increasingly hard to sustain as communities grow. While recent updates to Reddit's modqueue emphasize efficiency and reducing redundancy, recent research shows that moderators use the interface in varied ways, value objectives beyond throughput (such as fairness and accuracy), and often resist features that disrupt workflows. In this paper, we survey 106 active Reddit moderators to examine the objectives they bring to their modqueue work and the kinds of interventions they consider helpful. Our findings highlight wide variation in values and workflows, with no single objective beyond accuracy dominating, and different perspectives on which interventions are useful. To address this diversity, we introduce a simulation-based approach that can complement empirical findings by probing tradeoffs and testing potential interventions, and provide design recommendations based on our findings.

Figures (5)

• Figure 1: Moderator ratings of different report review objectives (5-point Likert scale). Accuracy received the highest average importance rating ($\mu=4.35$), followed by Fairness ($\mu=3.81$) and Wellbeing ($\mu=3.68$). Objectives such as Consensus, Toxicity Exposure, and Efficiency showed greater variation and lower mean ratings, and Redundancy had the lowest average importance rating overall ($\mu=2.10$). These patterns highlight differences in how moderators prioritize different goals during modqueue work.
• Figure 2: (left) The number of collisions increases as the number of active mods increase. (right) The optimal completion time of the modqueue decreases as the number of active mods increase. Notice that completion time of our baseline does not improve as dramatically with larger moderation teams as is the case for the optimal completion time values.
• Figure 3: (left) Average number of reports completed by an individual mod during a trial compared to the average reports seen; percentages describe the average percentage of reports seen that end up completed. (right) Average amount of time an individual mod spent working compared to the amount of time that actually contributes towards realized work (i.e. finished reports), or non-redundant time; percentage values indicate the percentage of total working time that is spent non-redundantly. Note that mods are spending time on and seeing more reports than those they complete, indicating that much of their actual effort is spent doing work that could be seen as "unnecessary" as it does not contribute to completing reports.
• Figure 4: (left) Increasing collision awareness reduces the number of collisions. (right) Completion time also improves with higher $\rho$, reaching optimal values when $\rho = 1$.
• Figure 5: (left) Reverse and Random traversal strategies reduce the number of collisions. (right) These same strategies also lead to faster completion times, with Random achieving performance close to the theoretical optimum.