Scrolly2Reel: Retargeting Graphics for Social Media Using Narrative Beats

TL;DR

The paper tackles the problem of retargeting scrollytelling content for social media reels by introducing narrative beats as a fundamental unit that governs alignment between narration and visuals. It contributes a Chrome DevTools-based system, Scrolly2Reel, which identifies beat boxes in a scrolly, hides them, and uses a beat-based script to automatically scroll and narrate, with optional GPT-driven shortening to adjust pacing. Quantitative and qualitative evaluations across 17 scrollies (producing 68 reels) show that fast-paced, beat-aligned videos yield the best pacing and, to a degree, alignment, while shortening beats with GPT generally improves pacing. The work demonstrates that beat-driven retargeting can preserve narrative coherence while adapting content to a shorter, more engaging medium, potentially expanding reach to younger audiences and enabling reuse of journalism graphics at scale. The key measurable mechanism is the computed scrolling speed, defined as $ ext{scrolling speed} = rac{ ext{end position} - ext{start position}}{ ext{estimated speaking time}} $, which ties beat length to narration duration and visual pacing.

Abstract

Content retargeting is crucial for social media creators. Once great content is created, it is important to reach as broad an audience as possible. This is particularly important in journalism where younger audiences are shifting away from print and towards short-video platforms. Many newspapers already create rich graphics for the web that they want to be able to reuse for social media. One example is scrollytelling sequences or "scrollies" -- immersive articles with graphics like animation, charts, and 3D visualizations that appear as a user scrolls. We present a system that helps transform scrollies into social media videos. By using the scriptwriting concept of narrative beats to extract fundamental storytelling units, we can create videos that are more aligned with narration, and allow for better pacing and stylistic changes. Narrative beats are thus an important primitive to retargeting content that matches the style of a new medium while maintaining the cohesiveness of the original content.

Figures (2)

• Figure 1: Scrolly2Reel provides an approach to convert scrollies in articles to a social media video with beat alignment. The example above shows a New York Times scrolly Dooley2023AmmunitionPlant
• Figure 2: Beat sequences with scripts and keyframes across different reel versions, taken from a Washington Post scrolly WP2022GasPrices