Towards High-Fidelity Synthetic Multi-platform Social Media Datasets via Large Language Models

TL;DR

This work addresses the scarcity of multi-platform social media data by proposing a framework to generate synthetic, cross-platform posts using large language models. It introduces Multi-Platform Topic Modeling (MPTM) prompting, which builds topic-aware sample pools across platforms and prompts LLMs to create platform-agnostic posts that reflect shared topics. The study evaluates fidelity across lexical features, sentiment, topics, embeddings, and named entities using two diverse datasets (election-related and influencer content) and three LLMs (GPT-4o, Gemini-2.0, Claude-3.5), revealing that cross-platform topic coherence and embedding fidelity are generally promising but model-dependent, with clear biases in hashtags, emojis, and sentiment. The findings suggest that with careful prompting and post-processing, realistic multi-platform synthetic datasets can support reproducibility and broaden access for researchers while mitigating platform-access constraints, though privacy and utility considerations warrant further investigation.

Abstract

Social media datasets are essential for research on a variety of topics, such as disinformation, influence operations, hate speech detection, or influencer marketing practices. However, access to social media datasets is often constrained due to costs and platform restrictions. Acquiring datasets that span multiple platforms, which is crucial for understanding the digital ecosystem, is particularly challenging. This paper explores the potential of large language models to create lexically and semantically relevant social media datasets across multiple platforms, aiming to match the quality of real data. We propose multi-platform topic-based prompting and employ various language models to generate synthetic data from two real datasets, each consisting of posts from three different social media platforms. We assess the lexical and semantic properties of the synthetic data and compare them with those of the real data. Our empirical findings show that using large language models to generate synthetic multi-platform social media data is promising, different language models perform differently in terms of fidelity, and a post-processing approach might be needed for generating high-fidelity synthetic datasets for research. In addition to the empirical evaluation of three state of the art large language models, our contributions include new fidelity metrics specific to multi-platform social media datasets.

Figures (8)

• Figure 1: Multi Platform Topic Modeling (MPTM)-based prompting approach for generating and evaluating synthetic social media datasets. Posts from different platforms from two datasets are first processed using topic models to identify shared topics. Based on these topics, we create sample pools for each platform. From these pools, we randomly select examples to use for LLM prompting. For comparison, our baseline method (Per-Platform prompting, presented in our previous work tari2024leveraging) selects random examples directly from each platform. This baseline approach is illustrated with dotted lines.
• Figure 2: Topic Overlap between real and synthetic data generated using MPTM prompting on the Elections dataset(top row) and Influencers dataset (bottom row). Topic overlap is better in synthetic influencers dataset than election dataset.
• Figure 3: Topic overlap among platforms in per-platform prompting (as proposed by Tari et al.tari2024leveraging) between real and synthetic data. (a) US Mid-Term Election Dataset (b) Dutch Influencer's Dataset. Topics on Facebook are better represented in synthetic datasets generated by all LLMs. Gemini-2.0 gives better topic overlap on TikTok and YouTube.
• Figure 4: Comparison of topics generated by different LLMs using MPTM prompting using dimensionally reduced topic embedding for (a) US Election Dataset (b) Dutch Influencer's Dataset.
• Figure 5: t-SNE visualizations of US and Dutch datasets across different social media platforms.