DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing

TL;DR

DynaVis tackles the challenge of visualizations with a large editing option space by blending natural language interfaces with dynamically synthesized GUI widgets. It introduces a widget synthesis engine driven by LLMs to generate both visual edits (Vega-Lite) and executable UI controls that persist for iterative refinement, enabling immediate visual feedback and fine-grained adjustments. A three-stage pipeline (data-grounded pre-processing, LLM-based synthesis, and program-analysis post-processing) ensures reliable chart updates and widget functionality. A within-subject user study (n=24) shows that DW (dynamic widgets) reduces editing effort, increases perceived control, and improves task ease compared to an NL-only baseline, though widget proliferation can overwhelm users in long sessions. Overall, DynaVis demonstrates how hybrid NL and dynamic UI widgets can lower the gulf of execution in visualization editing while preserving the explorability and feedback that static interfaces lack, with broad implications for no-code visualization and beyond.

Abstract

Users often rely on GUIs to edit and interact with visualizations - a daunting task due to the large space of editing options. As a result, users are either overwhelmed by a complex UI or constrained by a custom UI with a tailored, fixed subset of options with limited editing flexibility. Natural Language Interfaces (NLIs) are emerging as a feasible alternative for users to specify edits. However, NLIs forgo the advantages of traditional GUI: the ability to explore and repeat edits and see instant visual feedback. We introduce DynaVis, which blends natural language and dynamically synthesized UI widgets. As the user describes an editing task in natural language, DynaVis performs the edit and synthesizes a persistent widget that the user can interact with to make further modifications. Study participants (n=24) preferred DynaVis over the NLI-only interface citing ease of further edits and editing confidence due to immediate visual feedback.

Figures (11)

• Figure 1: DynaVis dynamically synthesizes widgets based on natural language commands for visualization editing. The user can describe an edit to the visualization, and DynaVis modifies the visualization and synthesizes a dynamic widget which the user can use for further edits (shown as $a \rightarrow b$). Alternatively, the user can directly ask for a dynamic widget to perform edits (shown as $a \rightarrow c$).
• Figure 2: (1) Alice asks DynaVis to create a line chart to show the stock trend by providing a natural language prompt. (2) She then asks DynaVis to edit the legend position by giving a natural language command
• Figure 3: (1) Alice asks DynaVis to increase the $x$-axis label text size to 20 and rotate it by 45 degrees counter-clockwise. (2) DynaVis edits the chart and also adds a dynamically synthesized widget for Alice to make further changes in the future.
• Figure 4: (1) This time Alice explicitly requests for a widget to change the color of each stock symbol by clicking the "+" button and giving the natural language prompt. (2) DynaVis synthesizes a widget, without making any edits to the chart.
• Figure 5: A dynamic widget is comprised of two components --- (1) The HTML script defining the UI (2) The JS callback function that listens for the changes in the UI to edit the visualization.