Narrative Player: Reviving Data Narratives with Visuals

TL;DR

Narrative Player tackles the challenge of transforming data narratives into engaging, data-driven videos by automatically extracting contextualized data facts from paragraphs and data tables and then mapping these facts to coherent visualization sequences optimized for consistency and context. The system comprises two modules: Narrative Analysis, which uses LLMs and sentence embeddings to parse long narratives into data facts, and Visual Generation, which formulates a context-aware visualization sequence via an optimization objective that balances transition costs, visual focus, and primary visualization activation, and then renders a data video with transitions and audio narration. A formal data fact representation as a 6-tuple ($fact := \{ type, parameters, measures, context, breakdowns, focus \}$) and a global objective $\mathcal{F} = \omega_1\mathcal{T} + \omega_2\mathcal{B} + \omega_3\mathcal{P}$ guide the mapping and sequencing of visuals, with $\mathcal{T}$ encoding transition costs, $\mathcal{B}$ a visual focus bonus, and $\mathcal{P}$ a primary-visual activation term. The approach is evaluated via user studies and expert studies, showing that automatically generated data videos are well-received and comparable to human-made videos, and ablations demonstrate the contributions of both narration analysis and visual generation modules. The work advances data storytelling by enabling end-to-end generation of consistent, contextualized, animated data narratives, with practical implications for reading experience, comprehension, and engagement in data-rich documents.

Abstract

Data-rich documents are commonly found across various fields such as business, finance, and science. However, a general limitation of these documents for reading is their reliance on text to convey data and facts. Visual representation of text aids in providing a satisfactory reading experience in comprehension and engagement. However, existing work emphasizes presenting the insights of local text context, rather than fully conveying data stories within the whole paragraphs and engaging readers. To provide readers with satisfactory data stories, this paper presents Narrative Player, a novel method that automatically revives data narratives with consistent and contextualized visuals. Specifically, it accepts a paragraph and corresponding data table as input and leverages LLMs to characterize the clauses and extract contextualized data facts. Subsequently, the facts are transformed into a coherent visualization sequence with a carefully designed optimization-based approach. Animations are also assigned between adjacent visualizations to enable seamless transitions. Finally, the visualization sequence, transition animations, and audio narration generated by text-to-speech technologies are rendered into a data video. The evaluation results showed that the automatic-generated data videos were well-received by participants and experts for enhancing reading.

Figures (7)

• Figure 1: An animated visual sequence example with seamless transitions, audio and subtitles automatically generated by Narrative Player from data narrative and data table for engaging reading experience, where means the visuals will transition to the next when the narration moves forward to the corresponding $x$-th segment.
• Figure 2: Narrative Player system overview and processing pipeline. The Narrative Analysis module automatically processes user-provided narration text and tabular data to generate fact candidates which the Visual Generation module then uses to create data videos.
• Figure 3: An illustrating case describing how to extract, validate, and select data fact candidates. In one LLM session for preliminary extraction, three facts (F1, F2, F3) have been extracted for one clause (C). Another LLM session with appropriate prompts rewrites the target clause based on the three facts and context into three rewritten clauses (C1, C2, C3). The sentence embedding model generates embeddings ($E_{c_{i}}$ and $E_{c}$) for all the clauses. Finally, F3 was ranked first based on the cosine similarity.
• Figure 4: An illustrating case of inferring vague clause within context. The narrative analysis module first detects keywords, "chill" and "winter", relating them to five candidate properties about temperature and three candidate values for months based on the narration and data. Then the module selects two clear clauses around as references, inferring two sets of referred properties and values, as shown in dashed boxes and arrows. The module further merges the intersection of them as filtered properties and values and extracts three fact candidates (F1, F2, F3).
• Figure 5: Two examples automatically generated by Narrative Player, each featuring a visualization sequence with corresponding transition animations. Transition animations are activated when the audio narration hits the relevant segments, with representative transitions highlighted in the text. More examples can be found on https://datavideos.github.io/Narrative_Player/.