Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PlotGen-Bench: Evaluating VLMs on Generating Visualization Code from Diverse Plots across Multiple Libraries

Yi Zhao, Zhen Yang, Shuaiqi Duan, Wenmeng Yu, Zhe Su, Jibing Gong, Jie Tang

TL;DR

PlotGen-Bench addresses the problem of evaluating vision-language models on generating executable visualization code from plots, including 3D, animated, and cross-library scenarios. It introduces a large-scale benchmark spanning 28 plot types across 5 libraries and 3 tasks, with a hybrid evaluation pipeline that combines automatic checks and VLM-as-a-judge. Experimental results show open-source VLMs lag in visual fidelity despite reasonable executability, while closed-source models show stronger cross-library generalization; performance also degrades on transformation and animation tasks, underscoring the need for execution-aware, multi-library training. The benchmark and accompanying data/code provide a foundation for advancing reliable, semantically faithful visualization code synthesis.

Abstract

Recent advances in vision-language models (VLMs) have expanded their multimodal code generation capabilities, yet their ability to generate executable visualization code from plots, especially for complex 3D, animated, plot-to-plot transformations, or multi-library scenarios, remains underexplored. To address this gap, we introduce PlotGen-Bench, a comprehensive benchmark for evaluating plot-to-code generation under realistic and complex visualization scenarios. The benchmark spans 9 major categories, 30 subcategories, and 3 core tasks-plot replication, plot transformation, and multi-library generation, covering both 2D, 3D and animated plots across 5 widely used visualization libraries. Through systematic evaluation of state-of-the-art open- and closed-source VLMs, we find that open-source models still lag considerably behind in visual fidelity and semantic consistency, despite achieving comparable code executability. Moreover, all models exhibit substantial degradation on reasoning-intensive tasks such as chart type conversion and animation generation. PlotGen-Bench establishes a rigorous foundation for advancing research toward more capable and reliable VLMs for visualization authoring and code synthesis, with all data and code available at https://plotgen.github.io.

PlotGen-Bench: Evaluating VLMs on Generating Visualization Code from Diverse Plots across Multiple Libraries

TL;DR

PlotGen-Bench addresses the problem of evaluating vision-language models on generating executable visualization code from plots, including 3D, animated, and cross-library scenarios. It introduces a large-scale benchmark spanning 28 plot types across 5 libraries and 3 tasks, with a hybrid evaluation pipeline that combines automatic checks and VLM-as-a-judge. Experimental results show open-source VLMs lag in visual fidelity despite reasonable executability, while closed-source models show stronger cross-library generalization; performance also degrades on transformation and animation tasks, underscoring the need for execution-aware, multi-library training. The benchmark and accompanying data/code provide a foundation for advancing reliable, semantically faithful visualization code synthesis.

Abstract

Recent advances in vision-language models (VLMs) have expanded their multimodal code generation capabilities, yet their ability to generate executable visualization code from plots, especially for complex 3D, animated, plot-to-plot transformations, or multi-library scenarios, remains underexplored. To address this gap, we introduce PlotGen-Bench, a comprehensive benchmark for evaluating plot-to-code generation under realistic and complex visualization scenarios. The benchmark spans 9 major categories, 30 subcategories, and 3 core tasks-plot replication, plot transformation, and multi-library generation, covering both 2D, 3D and animated plots across 5 widely used visualization libraries. Through systematic evaluation of state-of-the-art open- and closed-source VLMs, we find that open-source models still lag considerably behind in visual fidelity and semantic consistency, despite achieving comparable code executability. Moreover, all models exhibit substantial degradation on reasoning-intensive tasks such as chart type conversion and animation generation. PlotGen-Bench establishes a rigorous foundation for advancing research toward more capable and reliable VLMs for visualization authoring and code synthesis, with all data and code available at https://plotgen.github.io.
Paper Structure (34 sections, 27 figures, 6 tables)

This paper contains 34 sections, 27 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-language Models For Plot Code Generation
  4. Vision-language Models Code Generation Benchmark
  5. PlotGen-Bench
  6. Scope and Task Definition
  7. Data Construction
  8. Data Analysis
  9. Discussion
  10. Evaluation Metrics
  11. Experiments
  12. Evaluation Models
  13. Results on Difference Plot Types
  14. Results on Different Libraries
  15. Results on Plot Transformation
  16. ...and 19 more sections

Figures (27)

  • Figure 1: Comparison between regular figures on different libraries. and 3D, animation, interactive transformation on different models.
  • Figure 2: Plot subcategories distribution in PlotGen-Bench
  • Figure 3: Complexity Analysis of Code and Plots
  • Figure 4: The pipeline of PlotGen-Bench. PlotGen-Bench supports 5 Python rendering libraries, 3 tasks, and rendering validation across 30 types of visualizations, including complex plots such as 3D plots, animations, and interactive plots.
  • Figure 5: GPT-score of models at different complexity levels
  • ...and 22 more figures