Safe Spaces or Toxic Places? Content Moderation and Social Dynamics of Online Eating Disorder Communities

TL;DR

This study investigates how content moderation shapes discussions of eating disorders across Twitter/X, Reddit, and TikTok by combining network analysis with emotion and toxicity measures. Using retweet, subreddit-mention, and hashtag-cooccurrence networks, along with community detection via Louvain modularity, the authors map the social structure and thematic organization of ED-related content. They find that weaker moderation on Twitter fosters toxic pro-anorexia echo chambers, whereas TikTok and Reddit exhibit more recovery-oriented discourse embedded within mainstream topics due to stricter guardrails. The work highlights how moderation policies influence the formation and impact of online ED communities and offers guidance for designing safeguards that reduce harm while preserving supportive spaces. Overall, the paper contributes a cross-platform socio-technical framework for understanding and mitigating online mental-health harms driven by social dynamics and algorithmic exposure.

Abstract

Social media platforms have become critical spaces for discussing mental health concerns, including eating disorders. While these platforms can provide valuable support networks, they may also amplify harmful content that glorifies disordered cognition and self-destructive behaviors. While social media platforms have implemented various content moderation strategies, from stringent to laissez-faire approaches, we lack a comprehensive understanding of how these different moderation practices interact with user engagement in online communities around these sensitive mental health topics. This study addresses this knowledge gap through a comparative analysis of eating disorder discussions across Twitter/X, Reddit, and TikTok. Our findings reveal that while users across all platforms engage similarly in expressing concerns and seeking support, platforms with weaker moderation (like Twitter/X) enable the formation of toxic echo chambers that amplify pro-anorexia rhetoric. These results demonstrate how moderation strategies significantly influence the development and impact of online communities, particularly in contexts involving mental health and self-harm.

Figures (7)

• Figure 1: Screencaps of searches for anorexia-related content. (a) Searching for "anatwt" a term used by the pro-anorexia community on Twitter, returned posts promoting food restriction. (b) Search for "ana" was blocked on TikTok, redirecting users to mental health resources. (c) Search for "proana" on Reddit directs users to discussions in different forums.
• Figure 2: Hashtag co-occurrence network on (top) Twitter and (bottom) TikTok. Nodes are popular hashtags and edges link hashtags that are frequently used together. Node colors represent discovered communities and 15% of the labels are shown.
• Figure 3: Communities in the retweet network of Twitter. (Left) User network showing retweets between individual users. Colors correspond to different communities identified by the Louvain method. (Right) Chord diagram showing retweets within and between communities. The size of each chord represents the number of times members of a community retweeted themselves (self-retweeting), while the width of links shows the number of times they retweeted other communities.
• Figure 4: Subreddit mention network. Nodes represent subreddits, with node colors representing higher-level clusters. Links share the same color as the source subreddit, while node sizes are proportional to their degrees, indicating the level of interaction. Description of each cluster provided by Gemini 1.5 summarizing 200 randomly sampled posts from each cluster.
• Figure 5: Community emotion analysis. For each (a) community on Twitter, (b) Reddit discussion forum, or (c) TikTok hashtag, we aggregate the emotion confidence scores across all original tweets/submissions (open bars) and replies/comments (hashed bars) and show the share of posts with each emotion.