Reranking Social Media Feeds: A Practical Guide for Field Experiments
Tiziano Piccardi, Martin Saveski, Chenyan Jia, Jeffrey Hancock, Jeanne L. Tsai, Michael S. Bernstein
TL;DR
The paper tackles the challenge of assessing causal effects of social media feed ranking without platform cooperation by advocating a browser-extension approach that intercepts and realigns feed content in real time. It details how to implement up-ranking, down-ranking, and content-editing interventions with attention to latency, alongside measurement frameworks that include ecological momentary assessments, longitudinal surveys, and engagement signals. Practical guidance covers implementation, recruitment, onboarding, notifications, and privacy, with an open-source Javascript toolkit to enable reproducible field experiments. This work enables independent researchers to audit feed algorithms and contribute to designing safer, more transparent online social spaces.
Abstract
Social media plays a central role in shaping public opinion and behavior, yet performing experiments on these platforms and, in particular, on feed algorithms is becoming increasingly challenging. This article offers practical recommendations to researchers developing and deploying field experiments focused on real-time re-ranking of social media feeds. This article is organized around two contributions. First, we overview an experimental method using web browser extensions that intercepts and re-ranks content in real-time, enabling naturalistic re-ranking field experiments. We then describe feed interventions and measurements that this paradigm enables on participants' actual feeds, without requiring the involvement of social media platforms. Second, we offer concrete technical recommendations for intercepting and re-ranking social media feeds with minimal user-facing delay, and provide an open-source implementation. This document aims to summarize lessons learned, provide concrete implementation details, and foster the ecosystem of independent social media research.