ChartReformer: Natural Language-Driven Chart Image Editing

TL;DR

ChartReformer addresses editing charts when the original data is unavailable by decomposing a chart image into a data table and visual attributes and then re-creating the edited chart from natural language prompts. It introduces ChartCraft, a large synthetic dataset of roughly $100{,}000$ chart-edit pairs, and a two-stage pipeline that first de-renders charts and then edits them via a replotter, without relying on plotting code. The work standardizes chart edits into four categories (style, layout, format, data-centric), demonstrates strong performance against a ChartLlama baseline on image fidelity and edit correctness, and highlights the potential to improve accessibility and adaptability of chart visualizations in real-world applications. Together, these contributions provide a scalable framework for NL-driven chart editing that does not require original data, enabling flexible, image-first chart customization with practical impact for data communication.

Abstract

Chart visualizations are essential for data interpretation and communication; however, most charts are only accessible in image format and lack the corresponding data tables and supplementary information, making it difficult to alter their appearance for different application scenarios. To eliminate the need for original underlying data and information to perform chart editing, we propose ChartReformer, a natural language-driven chart image editing solution that directly edits the charts from the input images with the given instruction prompts. The key in this method is that we allow the model to comprehend the chart and reason over the prompt to generate the corresponding underlying data table and visual attributes for new charts, enabling precise edits. Additionally, to generalize ChartReformer, we define and standardize various types of chart editing, covering style, layout, format, and data-centric edits. The experiments show promising results for the natural language-driven chart image editing.

Figures (6)

• Figure 1: Examples of chart editing results from our methods. In total, our methods define and cover four types of chart editing: style, layout, format and data-centric edit.
• Figure 2: A Chart Image and edit-prompt are taken as input by the ChartReformer model, which predicts visual attributes and data for the corresponding edited chart. A Replotter software takes in these predicted parameters and generates the edited chart-image
• Figure 3: Distribution of Samples for Style Edits across Chart Categories
• Figure 4: Qualitative Results for ChartLlama
• Figure 5: Qualitative Results for ChartReformer