Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cleaning Up the Streets: Understanding Motivations, Mental Models, and Concerns of Users Flagging Social Media Content

Alice Qian Zhang, Kaitlin Montague, Shagun Jhaver

TL;DR

The paper investigates why users flag content on social media, how they understand flagging processes, and what concerns they have about cognitive load and privacy. Through 25 semi-structured interviews and interpretive qualitative analysis, it reveals a three-stage mental model of flagging (before, during, after) and highlights substantial transparency gaps that shape user expectations and behavior. The study identifies motivations (ethical judgments, social pressure, collective duty, and expression) and concerns (identity-based harms, prioritization, privacy, retaliation), and proposes design opportunities to improve user understanding, privacy protection, and reduce cognitive burdens. By examining end-user labor in moderation, the work emphasizes shared responsibility between platforms and users and offers concrete recommendations for flagging interfaces, dashboards, and policy considerations to foster more transparent and effective online harm regulation.

Abstract

Social media platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors to reduce online harm. The deceptively simple flagging interfaces on nearly all major social media platforms disguise complex underlying interactions among users, algorithms, and moderators. Through interviewing 25 social media users with prior flagging experience, most of whom belong to marginalized groups, we examine end-users' understanding of flagging procedures, explore the factors that motivate them to flag, and surface their cognitive and privacy concerns. We found that a lack of procedural transparency in flagging mechanisms creates gaps in users' mental models, yet they strongly believe that platforms must provide flagging options. Our findings highlight how flags raise critical questions about distributing labor and responsibility between platforms and users for addressing online harm. We recommend innovations in the flagging design space that enhance user comprehension, ensure privacy, and reduce cognitive burdens.

Cleaning Up the Streets: Understanding Motivations, Mental Models, and Concerns of Users Flagging Social Media Content

TL;DR

The paper investigates why users flag content on social media, how they understand flagging processes, and what concerns they have about cognitive load and privacy. Through 25 semi-structured interviews and interpretive qualitative analysis, it reveals a three-stage mental model of flagging (before, during, after) and highlights substantial transparency gaps that shape user expectations and behavior. The study identifies motivations (ethical judgments, social pressure, collective duty, and expression) and concerns (identity-based harms, prioritization, privacy, retaliation), and proposes design opportunities to improve user understanding, privacy protection, and reduce cognitive burdens. By examining end-user labor in moderation, the work emphasizes shared responsibility between platforms and users and offers concrete recommendations for flagging interfaces, dashboards, and policy considerations to foster more transparent and effective online harm regulation.

Abstract

Social media platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors to reduce online harm. The deceptively simple flagging interfaces on nearly all major social media platforms disguise complex underlying interactions among users, algorithms, and moderators. Through interviewing 25 social media users with prior flagging experience, most of whom belong to marginalized groups, we examine end-users' understanding of flagging procedures, explore the factors that motivate them to flag, and surface their cognitive and privacy concerns. We found that a lack of procedural transparency in flagging mechanisms creates gaps in users' mental models, yet they strongly believe that platforms must provide flagging options. Our findings highlight how flags raise critical questions about distributing labor and responsibility between platforms and users for addressing online harm. We recommend innovations in the flagging design space that enhance user comprehension, ensure privacy, and reduce cognitive burdens.
Paper Structure (37 sections, 2 figures, 2 tables)

This paper contains 37 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Content Moderation Affordances on Social Media
  4. User Perspectives on Content Moderation
  5. Flagging Mechanisms on Social Media Platforms
  6. Methods
  7. Participant Recruitment
  8. Data Collection and Analysis
  9. Findings
  10. RQ1: What are users' mental models of how social media platforms implement flagging mechanisms, and how do they develop these models?
  11. Discovery of the option to flag content
  12. Review process for flagged content
  13. Updates on flagged content from platforms
  14. Moderation actions performed alongside flagging
  15. Misuse of flags
  16. ...and 22 more sections

Figures (2)

  • Figure 1: YouTube's flagging interface pages.
  • Figure 2: Facebook's support inbox dashboard showing posts a user has flagged.