Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

De-rendering, Reasoning, and Repairing Charts with Vision-Language Models

Valentin Bonas, Martin Sinnona, Viviana Siless, Emmanuel Iarussi

TL;DR

This work introduces a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design, and highlights the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design.

Abstract

Data visualizations are central to scientific communication, journalism, and everyday decision-making, yet they are frequently prone to errors that can distort interpretation or mislead audiences. Rule-based visualization linters can flag violations, but they miss context and do not suggest meaningful design changes. Directly querying general-purpose LLMs about visualization quality is unreliable: lacking training to follow visualization design principles, they often produce inconsistent or incorrect feedback. In this work, we introduce a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design. Our system reconstructs the structure of a chart from an image, identifies design flaws using vision-language reasoning, and proposes concrete modifications supported by established principles in visualization research. Users can selectively apply these improvements and re-render updated figures, creating a feedback loop that promotes both higher-quality visualizations and the development of visualization literacy. In our evaluation on 1,000 charts from the Chart2Code benchmark, the system generated 10,452 design recommendations, which clustered into 10 coherent categories (e.g., axis formatting, color accessibility, legend consistency). These results highlight the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design, opening the door to more intelligent and accessible authoring tools.

De-rendering, Reasoning, and Repairing Charts with Vision-Language Models

TL;DR

This work introduces a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design, and highlights the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design.

Abstract

Data visualizations are central to scientific communication, journalism, and everyday decision-making, yet they are frequently prone to errors that can distort interpretation or mislead audiences. Rule-based visualization linters can flag violations, but they miss context and do not suggest meaningful design changes. Directly querying general-purpose LLMs about visualization quality is unreliable: lacking training to follow visualization design principles, they often produce inconsistent or incorrect feedback. In this work, we introduce a framework that combines chart de-rendering, automated analysis, and iterative improvement to deliver actionable, interpretable feedback on visualization design. Our system reconstructs the structure of a chart from an image, identifies design flaws using vision-language reasoning, and proposes concrete modifications supported by established principles in visualization research. Users can selectively apply these improvements and re-render updated figures, creating a feedback loop that promotes both higher-quality visualizations and the development of visualization literacy. In our evaluation on 1,000 charts from the Chart2Code benchmark, the system generated 10,452 design recommendations, which clustered into 10 coherent categories (e.g., axis formatting, color accessibility, legend consistency). These results highlight the promise of LLM-driven recommendation systems for delivering structured, principle-based feedback on visualization design, opening the door to more intelligent and accessible authoring tools.
Paper Structure (8 sections, 3 figures)

This paper contains 8 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Framework
  4. Chart Deconstruction
  5. Recommended updates
  6. Interactive refinement
  7. Evaluation
  8. Conclusions and Future work

Figures (3)

  • Figure 1: System workflow overview. Starting from a raster bar chart (A), ChartCoder de-renders the figure into executable Matplotlib code (B; b1–b2)—the chart deconstruction stage. GPT-OSS then inspects the recovered code and provides recommended updates (C; c1), e.g., switching to a line chart to show trends and placing the legend outside the plot area. Finally, through interactive refinement (D; d1–d2), the code is iteratively edited and re-rendered, producing a clean vector visualization with the requested changes (E).
  • Figure 2: Tool flow and web interface overview. Users begin by uploading a visualization (A). The system then analyzes the input and presents recommendation options through the web UI (B), allowing users to explore alternative encodings and design choices. Finally, the selected modifications are applied, and the tool generates an updated visualization (C) reflecting the improvements.
  • Figure 3: Recommendation embedding space. Visualization recommendations are encoded into feature vectors and projected with UMAP to form a recommendation manifold (A). Similar issues cluster together, such as low image resolution, axis formatting, or colorblind-unfriendly palettes. Example visualizations (B) illustrate specific cases, including legend inconsistencies, small text sizes, and inadequate color contrast, highlighting how the system groups and presents related feedback.