Reviving Static Charts into Live Charts

TL;DR

This paper introduces Live Charts, a format that revives static charts by delivering sequential, multi-sensory data stories through synchronized animations and audio narration. It presents a fully automatic pipeline that first uses a dual-stream Graph Neural Network to recover underlying data and visual encodings from SVG charts, then leverages large language models to generate contextual narration and data-driven insights, which are paired with purpose-built animations. The authors validate their approach with a comprehensive evaluation including real-use cases, chart-element recognition performance, a crowd-sourced user study (N=90), and expert interviews, showing that Live Charts improve understandability, memorability, and focus compared to static charts, with animation providing additional benefits over plain narration. They discuss implications for accessibility, potential tool integrations, and the necessity of human–AI collaboration to handle diversity in user preferences and ensure data accuracy, outlining future directions for broader chart types and more flexible animation libraries.

Abstract

Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience.

Figures (7)

• Figure 1: An example of a Live Chart. (a1-a5) The sequential process of the Live Chart. (b1-b5) The animations in the chart. The texts below are the audio narration for the corresponding frames, with the first tag indicating the chart component and the insight type.
• Figure 2: Our automatic approach for generating Live Charts.
• Figure 3: A GNN-based approach to classify chart elements for a chart. (a) Original SVG-based chart; (b) Conversion from SVG to graph; (c) Two different vector encoders to extract features; (d) Classification of each element by a multi-layer perceptron. (e) Categories for elements in the chart. Each box corresponds to a primary category, followed by a set of sub-categories.
• Figure 4: The process of recovering visual encodings. (a) The example chart. Dotted dark blue lines connect chart elements with their SVG expressions. (b) Matching the mark with the legend. (c) Data calculation for different charts. (d) One path in the labeled SVG.
• Figure 5: Creating narration and animations involves multiple sub-tasks for GPT-3, depicted by the inner blue boxes and arrows. The starting position of the arrow indicates the information contained within the prompt (with "Data Table" being a necessary element for each prompt). Each blue box represents the output generated by GPT-3. (a1)-(a8) are part of prompt components. We use dotted gray lines and numbers to illustrate the prompt usage during the process. (b) The insight structure in JSON format. Sentences and phrases will be added in animation generation. Italic text with a blue background represents an example of the prompt output, using the "airport chart" in \ref{['fig:ChartUnder']}(a), to demonstrate how the process works.