Moving towards informative and actionable social media research

Abstract

Social media is nearly ubiquitous in modern life, raising concerns about its societal impacts-from mental health and polarization to violence and democratic disruption. Yet research on its causal effects remains inconclusive: observational studies often find concerning associations, while randomized controlled trials (RCTs) tend to yield small, conflicting, or null results. Literature summaries tend to causally prioritize findings from RCTs, often arguing that concerns about social media are overstated. However, like observational studies, RCTs rely on assumptions that can easily be violated in the context of social media, especially regarding societal outcomes at scale. Here, we enumerate and examine the features of social media as a complex system that challenge our ability to infer causality at societal scales. Drawing on insight from disciplines that have faced similar challenges, like climate-science or epidemiology, we propose a path forward that combines the strength of observational and experimental approaches while acknowledging the limitations of each.

Figures (2)

• Figure 1: Illustrations of key properties of complex systems that make some RCTs particularly difficult to interpret. A Global outcomes do not linearly depend on the sum of their individual components, making conclusions from small groups to larger societies generally difficult. B More specifically, the outcomes depend on the history of the system, known as hysteresis, which makes conclusions about any reverse effects impossible. C Feedback loops that prevent conclusions about one variable without considering the value of another at a specific point in time. D Spillover effects in network-structures, known as violations of the "stable unit treatment value assumption" (SUTVA), prevent a clean separation of treatment and control conditions.
• Figure 2: Illustrations of the triangulation approach.