Data Playwright: Authoring Data Videos with Annotated Narration

TL;DR

The paper addresses the challenge of producing data videos that narrate stories with synchronized animated visuals by introducing annotated narration, a unified NL-based format that embeds design authoring commands within narration. It presents Data Playwright, a prototype system whose automatic interpreter converts uploaded visualizations and annotated narration into audio narration, animations, and timing, enabling end-to-end data video synthesis with real-time preview and fine-tuning. A formative study informs the annotation syntax and the design of a robust interpreter, while a user study demonstrates that participants can effectively create data videos with annotated narration and derive meaningful, shareable example videos. The work advances data storytelling by combining narrative content with authoring intents, leveraging LLMs for element extraction and animation inference, and offering a cohesive platform that can be integrated into broader tools and workflows to democratize data video creation.

Abstract

Creating data videos that effectively narrate stories with animated visuals requires substantial effort and expertise. A promising research trend is leveraging the easy-to-use natural language (NL) interaction to automatically synthesize data video components from narrative content like text narrations, or NL commands that specify user-required designs. Nevertheless, previous research has overlooked the integration of narrative content and specific design authoring commands, leading to generated results that lack customization or fail to seamlessly fit into the narrative context. To address these issues, we introduce a novel paradigm for creating data videos, which seamlessly integrates users' authoring and narrative intents in a unified format called annotated narration, allowing users to incorporate NL commands for design authoring as inline annotations within the narration text. Informed by a formative study on users' preference for annotated narration, we develop a prototype system named Data Playwright that embodies this paradigm for effective creation of data videos. Within Data Playwright, users can write annotated narration based on uploaded visualizations. The system's interpreter automatically understands users' inputs and synthesizes data videos with narration-animation interplay, powered by large language models. Finally, users can preview and fine-tune the video. A user study demonstrated that participants can effectively create data videos with Data Playwright by effortlessly articulating their desired outcomes through annotated narration.

Figures (6)

• Figure 1: An annotated narration example. Users can incorporate authoring commands ({enclosed in blue curly brackets}) while crafting their text narration. They can also specify the desired duration of animations with orange square brackets ([ ]). The bottom portion shows the output video.
• Figure 2: A data video specification depicting the example in \ref{['fig: workflow']}.
• Figure 3: Automatic interpreter to synthesize data video components from users' annotated narration and visualizations.
• Figure 4: Data Playwright Interface: Users can preview and fine-tune the data video using NL (b), interactive widgets (d), and the code panel (e).
• Figure 5: Data video examples created in the user study from real-world storytelling practices.