Post Guidance for Online Communities

TL;DR

This paper introduces Post Guidance, a proactive, community-specific moderation paradigm that surfaces interventions as users draft posts. Through a large-scale randomized field experiment on Reddit (97,616 users, 33 subreddits, 63 days), the authors show that Post Guidance increases the share of non-removed contributions, reduces moderation workload, and enhances contribution quality, while overall posting activity and broad user engagement do not rise; effects are stronger in communities with heavier use of Post Guidance and AutoModerator. The approach blends user-centric nudges with rule-based interventions to educate contributors in real time, offering scalable benefits without imposing manual reviewer burdens. The findings suggest Post Guidance is adaptable to other platforms and content forms, with future work exploring richer rule sets, machine learning integration, and multimodal content support, while carefully balancing transparency and user experience.

Abstract

Effective content moderation in online communities is often a delicate balance between maintaining content quality and fostering user participation. In this paper, we introduce post guidance, a novel approach to community moderation that proactively guides users' contributions using rules that trigger interventions as users draft a post to be submitted. For instance, rules can surface messages to users, prevent post submissions, or flag posted content for review. This uniquely community-specific, proactive, and user-centric approach can increase adherence to rules without imposing additional burdens on moderators. We evaluate a version of Post Guidance implemented on Reddit, which enables the creation of rules based on both post content and account characteristics, via a large randomized experiment, capturing activity from 97,616 posters in 33 subreddits over 63 days. We find that Post Guidance (1) increased the number of ``successful posts'' (posts not removed after 72 hours), (2) decreased moderators' workload in terms of manually-reviewed reports, (3) increased contribution quality, as measured by community engagement, and (4) had no impact on posters' own subsequent activity, within communities adopting the feature. Post Guidance on Reddit was similarly effective for community veterans and newcomers, with greater benefits in communities that used the feature more extensively. Our findings indicate that post guidance represents a transformative approach to content moderation, embodying a paradigm that can be easily adapted to other platforms to improve online communities across the Web.

Figures (7)

• Figure 1: How Post Guidance works. When users try to write posts in a community, their contribution attempts are matched against a set of rules configured by the moderators of that community (left). Posts can only be submitted if they fulfill these rules, e.g., the post in the image cannot be submitted as it does not end with a question mark. Here, we present a user-level experiment ($n_{\text{users}}$=$97{,}616$) where users were either exposed to this feature ('treatment') or not ('control') across 33 communities on Reddit.
• Figure 2: Details about experiment enrollment. On the left, we show the number of users enrolled in the experiment per day; there is seasonality (with lower enrollment during weekends) and a continuous drop in the number of enrolled users in the first few weeks (as users can only enroll once). On the right, we show the fraction of users in the experiment per subreddit considered; there is a long tail of subreddits with less than 1% of users in the experiment. Note that subreddits are sorted by the number of distinct users that entered the posting interface during the study period (from largest to smallest).
• Figure 3: Timeline of the experiment. Users were enrolled in a 35-day period between 24 July 2023 and 28 August 2023. After enrolling, outcomes are calculated in a 28-day follow-up period. In the figure, we use gray lines to symbolize the tracking period of hypothetical users enrolling in different days.
• Figure 4:
• Figure 7: Depiction of the heterogeneity of the effect. For three of the outcomes considered (see Table \ref{['tab:outcomes']}), we show the distribution of subreddit-level effects with a kernel density estimate (KDE) plot. Close to the $x$-axis, we plot the actual effect sizes observed for each community using different markers for significant ($\times$) and not significant ($|$) subreddit-level effects. Note that effect sizes vary widely, going from negative to positive across all four outcomes (although, in many cases, the estimated effects are not statistically significant considering only one subreddit). To obtain the significance of the effects on a subreddit level, we repeatedly fit the Poisson Regression model depicted in Eq. \ref{['eq:1']} to the data of each of the 33 subreddits considered. The significance reported in Fig. \ref{['fig:het']} is associated with the $p$-value of coefficients $\beta$ associated with each regression.