Athanor: Authoring Action Modification-based Interactions on Static Visualizations via Natural Language

TL;DR

Athanor addresses the challenge of adding interactivity to static visualizations without access to source code or data by introducing a three-part pipeline: an action-modification design space, a multi-agent requirement analyzer, and a visualization abstraction translator. It converts static SVG visualizations into a constraint-based representation and uses multimodal language models to translate natural-language interaction requests into executable specifications, which are then applied through constraint-driven modifications. Two case studies and 11 user interviews demonstrate that the approach covers a wide range of user needs, preserves visual fidelity, and enables efficient authoring of interactions via natural language. The work offers a practical, implementation-agnostic framework for rapidly transforming static charts into interactive tools, with potential expansions to external data bindings and broader visualization types.

Abstract

Interactivity is crucial for effective data visualizations. However, it is often challenging to implement interactions for existing static visualizations, since the underlying code and data for existing static visualizations are often not available, and it also takes significant time and effort to enable interactions for them even if the original code and data are available. To fill this gap, we propose Athanor, a novel approach to transform existing static visualizations into interactive ones using multimodal large language models (MLLMs) and natural language instructions. Our approach introduces three key innovations: (1) an action-modification interaction design space that maps visualization interactions into user actions and corresponding adjustments, (2) a multi-agent requirement analyzer that translates natural language instructions into an actionable operational space, and (3) a visualization abstraction transformer that converts static visualizations into flexible and interactive representations regardless of their underlying implementation. Athanor allows users to effortlessly author interactions through natural language instructions, eliminating the need for programming. We conducted two case studies and in-depth interviews with target users to evaluate our approach. The results demonstrate the effectiveness and usability of our approach in allowing users to conveniently enable flexible interactions for static visualizations.

Figures (8)

• Figure 1: Users can upload existing visualizations in the chart view and present their authoring requirements (a) in the dialogue view. The multi-agent requirement analyzer employs translation, correction, and guidance agents that work collaboratively to translate users' requirements into specifications defined in (b) is the action-modification design space. The uploaded visualizations are abstracted to constraint-based presentations, and the analyzed modifications are also represented through constraints (c). Driven by these constraints, the visualization becomes interactive.
• Figure 2: The design space of visualization modifications.
• Figure 3: The process of LLM parsing users natural language input.
• Figure 4: The constraint model liu2024spatial is employed as the representation.
• Figure 5: The interface of the visualization interaction authoring system. On the left is the chart view; on the right is the dialogue view.