Comprehending Spatio-temporal Data via Cinematic Storytelling using Large Language Models

TL;DR

The paper addresses the challenge of making spatio-temporal data comprehensible to broad audiences by introducing MapMuse, a cinematic storytelling framework that leverages large language models, retrieval augmented generation, and agentic LLM workflows. It demonstrates two case studies on Porto's taxi data: a heatmap-based narrative with numerous POIs and a single-trajectory narrative, highlighting both the storytelling potential and hallucination risks. Key contributions include a cinema-inspired, three-act narrative framework for data, an end-to-end LLM-driven workflow with validation to mitigate errors, and a discussion of open problems and future research directions. The work aims to enhance comprehension, engagement, and actionable insight for spatio-temporal data in urban mobility and related domains, while acknowledging current limitations and the need for further benchmarking and tool refinement.

Abstract

Spatio-temporal data captures complex dynamics across both space and time, yet traditional visualizations are complex, require domain expertise and often fail to resonate with broader audiences. Here, we propose MapMuse, a storytelling-based framework for interpreting spatio-temporal datasets, transforming them into compelling, narrative-driven experiences. We utilize large language models and employ retrieval augmented generation (RAG) and agent-based techniques to generate comprehensive stories. Drawing on principles common in cinematic storytelling, we emphasize clarity, emotional connection, and audience-centric design. As a case study, we analyze a dataset of taxi trajectories. Two perspectives are presented: a captivating story based on a heat map that visualizes millions of taxi trip endpoints to uncover urban mobility patterns; and a detailed narrative following a single long taxi journey, enriched with city landmarks and temporal shifts. By portraying locations as characters and movement as plot, we argue that data storytelling drives insight, engagement, and action from spatio-temporal information. The case study illustrates how MapMuse can bridge the gap between data complexity and human understanding. The aim of this short paper is to provide a glimpse to the potential of the cinematic storytelling technique as an effective communication tool for spatio-temporal data, as well as to describe open problems and opportunities for future research.

Figures (4)

• Figure 1: Heat-map of taxi trip destinations in Porto.
• Figure 2: Points of interest (POIs) in Porto, mentioned in the storytelling narratives for the taxi endpoints heat-map.
• Figure 3: Case study: single trajectory with hallucinations.
• Figure 4: Overview of the MapMuse agentic LLM workflow.