Characterizing LLM-driven Social Network: The Chirper.ai Case

TL;DR

This work investigates how LLM-driven social networks differ from human-driven ones by comparing Chirper.ai, a platform populated entirely by AI agents, with Mastodon, a human-driven baseline. Using large-scale data (65k+ Chirpers with 7.7M AI-generated posts vs 117k Mastodon users with 16M posts), the study analyzes posting behavior, self-disclosure, abusive content, detectability of AI-generated text, and network structure. Key findings show Chirper posts are longer and more emoji-rich, with frequent yet often hallucinated mentions, and agents disclose personal information at higher rates aligned with their prompts; abusive content exists and correlates with higher engagement, while zero-shot detectors struggle to reliably distinguish AI from human text. Network analysis reveals a broadly connected but sparsely clustered Chirper graph, with abusive actors occupying central positions, suggesting moderation strategies should target structural features in addition to content. Overall, the paper highlights both the potential and risks of AI-driven social ecosystems and provides a baseline for moderation and detection in AI-mediated networks.

Abstract

Large language models (LLMs) demonstrate the ability to simulate human decision-making processes, enabling their use as agents in modeling sophisticated social networks, both offline and online. Recent research has explored collective behavioral patterns and structural characteristics of LLM agents within simulated networks. However, empirical comparisons between LLM-driven and human-driven online social networks remain scarce, limiting our understanding of how LLM agents differ from human users. This paper presents a large-scale analysis of Chirper.ai, an X/Twitter-like social network entirely populated by LLM agents, comprising over 65,000 agents and 7.7 million AI-generated posts. For comparison, we collect a parallel dataset from Mastodon, a human-driven decentralized social network, with over 117,000 users and 16 million posts. We examine key differences between LLM agents and humans in posting behaviors, abusive content, and social network structures. Our findings provide critical insights into the evolving landscape of online social network analysis in the AI era, offering a comprehensive profile of LLM agents in social simulations.

Figures (8)

• Figure 1: CDF plots of text length, number of emojis, hashtags, and mentions used in social submissions between Chirper.ai and Mastodon.
• Figure 2: CDF plots of self-disclosing posts ratio and a number of unique self-disclosures across accounts.
• Figure 3: Proportion of abusive submissions in the 5 prominent categories on Chirper.ai and Mastodon.
• Figure 4: Proportion of abusive submissions or backstories in the 5 prominent categories produced by non-abusive descriptions. The numbers in the bracket denote the total volume of abusive submissions and backstories in corresponding category.
• Figure 5: Comparison on the number of comments between posts containing and excluding abusive content in the 5 prominent categories respectively. The statistical significance is reported by the Mann-Whitney U test. *: $p<0.05$; ***: $p<0.001$; ****: $p<0.0001$.