Incorporating Procedural Fairness in Flag Submissions on Social Media Platforms

TL;DR

This study examines how four flagging design components—classification granularity, posting guidelines, a free-text input, and moderator type—influence flaggers’ perceptions of procedural fairness in social media moderation. Through a large randomized experiment (N=2,936) across 54 flagging scenarios, the authors find that including posting guidelines increases perceived transparency and that a text box for open-ended input boosts perceived voice, while classification granularity and moderator identity have limited effects. Qualitative analysis reveals user desires for greater expressivity, timely outcome notifications, better transparency about review procedures, and safeguards against flagging abuse. The findings yield concrete design recommendations for fairer flagging systems that reduce cognitive burden while enhancing user engagement and trust in moderation processes.

Abstract

Flagging mechanisms on social media platforms allow users to report inappropriate posts/accounts for review by content moderators. These reports are pivotal to platforms' efforts toward regulating norm violations. This paper examines how platforms' design choices in implementing flagging mechanisms influence flaggers' perceptions of content moderation. We conducted a survey experiment asking US respondents (N=2,936) to flag inappropriate posts using one of 54 randomly assigned flagging implementations. After flagging, participants rated their fairness perceptions of the flag submission process along the dimensions of consistency, transparency, and voice (agency). We found that participants perceived greater transparency when flagging interfaces included community guidelines and greater voice when they incorporated a text box for open-ended feedback. Our qualitative analysis highlights user needs for improved accessibility, educational support for reporting, and protections against false flags. We offer design recommendations for building fairer flagging systems without exacerbating the cognitive burden of submitting flags.

Figures (4)

• Figure 1: When users initiate the flagging process on social media platforms, Instagram mandates that they specify the rule violation of the post (left), Facebook offers a link to its community guidelines (middle), and YouTube provides a text box to elaborate on the specific reasons for flagging content (right).
• Figure 2: This example illustrates Facebook's flag classification scheme after users initiate the flag submission process. In this example, the user categorizes her reason for flagging as 'Violence, hate or exploitation' from the main menu; this selection displays a submenu from which the user selects the subcategory 'Credible threat to safety.' Finally, the system displays these selections to the user, along with the submit button that prompts completing the report.
• Figure 3: Participants can express their opinions about the flagged post using the text entry feature provided by Qualtrics.
• Figure 4: The four components, the different levels of each component, and an example scenario of the flagging mechanism experienced by participants. The left column outlines each element of the flagging mechanism along with its corresponding levels, while the right column exemplifies a scenario comprising a randomly selected combination of these levels.