Large Language Models Help Reveal Unhealthy Diet and Body Concerns in Online Eating Disorders Communities

TL;DR

The paper tackles the challenge of identifying unhealthy online ED communities that use obfuscated language by proposing a framework that aligns open-source LLMs to the linguistic patterns of specific ED communities. It builds a large-scale data pipeline from 2.6 million tweets, detects 402 communities via retweet networks, and selects the 20 largest for detailed analysis. By fine-tuning Llama-3 on community posts, the authors create community proxies that are then evaluated against psychometric instruments (SWED) to reveal varying ED risk across communities, with Pro-ED showing the highest risk and Anti-ED showing lower risk. The approach yields robust cross-validation across classification, toxicity, emotion, embedding similarity, and human judgments, offering a scalable tool for public health monitoring and targeted interventions, while acknowledging limitations around coverage, prompts, biases, and ethical considerations.

Abstract

Eating disorders (ED), a severe mental health condition with high rates of mortality and morbidity, affect millions of people globally, especially adolescents. The proliferation of online communities that promote and normalize ED has been linked to this public health crisis. However, identifying harmful communities is challenging due to the use of coded language and other obfuscations. To address this challenge, we propose a novel framework to surface implicit attitudes of online communities by adapting large language models (LLMs) to the language of the community. We describe an alignment method and evaluate results along multiple dimensions of semantics and affect. We then use the community-aligned LLM to respond to psychometric questionnaires designed to identify ED in individuals. We demonstrate that LLMs can effectively adopt community-specific perspectives and reveal significant variations in eating disorder risks in different online communities. These findings highlight the utility of LLMs to reveal implicit attitudes and collective mindsets of communities, offering new tools for mitigating harmful content on social media.

Figures (5)

• Figure 1: Communities in the retweet network. User network showing retweets between individual users. Colors correspond to different communities identified by the Louvain method.
• Figure 2: The framework of our method. (1) We align an LLM (Llama-3) to the language and mindset of an ED community. The alignment is achieved by finetuning the LLM to generate tweets written by users in the community by following instructions. (2) To prove that the alignment is effective, we focus on three sets of tweets: $\alpha$. human-written tweets, $\beta$. vanilla (unfinetuned) LLM-generated tweets, and $\gamma$. finetuned LLM-generated tweets. We show that $\gamma$ is closer to $\alpha$ than $\beta$ is, from the following aspects: (a) A classifier trained to classify the community origins of $\alpha$ performs equally well on $\gamma$, but not on $\beta$; (b) the emotion and toxicity distributions of $\gamma$ are much closer to that of $\alpha$ compared to that of $\beta$ are; (c) the embeddings of $\gamma$ are closer to that of $\alpha$ in the embedding space than that of $\beta$ are; (d) the human annotator decides that $\gamma$ is more aligned to underlying distribution of $\alpha$ than $\beta$ is. (3) As the LLM is aligned with the community and can speak as a typical individual in the community, we administer an eating disorder questionnaire to it and aim to screen the community for ED.
• Figure 3: Toxicity distribution across different communities of human-written posts, vanilla LLM generated posts, and finetuned LLM generated posts.
• Figure 4: Emotion distributions from different communities of (a) human-written posts, (b) vanilla LLM generated posts, and (c) finetuned LLM generated posts.
• Figure 5: Wordclouds of popular terms appearing in the original tweets posted within each community.