ChartEditor: A Human-AI Paired Tool for Authoring Pictorial Charts

TL;DR

This work introduces ChartEditor, a human-AI paired tool that converts basic charts into pictorial charts by decomposing charts into a hierarchical Chart Tree, translating user intent through natural language prompts, and enabling interactive refinement. It couples a chart decomposition pipeline trained on a new ChartSS dataset with a diffusion-based automatic generation module (GLIGEN) to produce contextually relevant pictorial elements. The approach is evaluated both quantitatively (segmentation accuracy, style transfer metrics) and through a user study comparing ChartEditor with existing tools, demonstrating improved usability, efficiency, and design fidelity. The results suggest ChartEditor effectively balances automation and user control, reducing design time while preserving data integrity, with future work extending chart types and customization capabilities.

Abstract

Pictorial charts are favored for their memorability and visual appeal, offering a more engaging alternative to basic charts. However, their creation can be complex and time-consuming due to the lack of native support in popular visualization tools like Tableau. While AI-generated content (AIGC) tools have lowered the barrier to creating pictorial charts, they often lack precise design control. To address this issue, we introduce ChartEditor, a human-AI paired tool that transforms basic charts into pictorial versions based on user intent. ChartEditor decomposes chart images into visual components and organizes them within a hierarchical tree. Based on this tree, users can express their intent in natural language, which is then translated into modifications to the hierarchy. In addition, users can directly interact with and modify specific chart components via an intuitive interface to achieve fine-grained design control. A user study demonstrates the effectiveness and usability of ChartEditor in simplifying the creation of pictorial charts.

Figures (8)

• Figure 1: Comparison of Pictorial Chart Generation Methods. (a) Manual creation with Adobe Illustrator requires extensive collaboration and iterative revisions to produce high-quality charts. (b) AI-generated methods offer quick and visually appealing charts but distort the original data. (c) $\mathsf{ChartEditor}$ balances automation and user control, enabling efficient, accurate, and customizable chart generation.
• Figure 2: $\mathsf{ChartEditor}$ transforms a basic chart into its pictorial version in three steps. (1) Chart Decomposition: The input chart is broken down into fundamental visual elements, organized within a hierarchical "Chart Tree" to enable structured editing. (2) Automatic Generation: Users provide prompts, and AI generates contextually relevant pictorial elements, such as icons and background elements, that integrate with the basic chart structure. (3) Interactive Refinement: Users refine the chart by directly modifying components within the Chart Tree or the chart image itself, ensuring precise adjustments and maintaining data integrity. The highlighted parts in the Chart Tree indicate components that have been identified or modified during the respective step.
• Figure 3: An illustrative example of how the chart tree facilitates the creation of pictorial charts. The first layer (white) represents the decomposition of the chart into its components. The second layer (pink) illustrates the replacement of these components during the auto-generation phase with icons or images imbued with semantic information. Finally, the components are recombined to create the final pictorial chart.
• Figure 4: Common design patterns when applying pictorial objects to marks and axes in bar charts, pie charts, and line charts.
• Figure 5: Comparison of chart segmentation results between Semantic-SAM li2023semantic and our method. (a) Shows the original basic charts, (b) Displays the segmentation results from Semantic-SAM, and (c) Illustrates the segmentation results from our approach, highlighting improvements in identifying and labeling key chart elements.

Theorems & Definitions (1)

• Example 1: An Example of Chart Tree