SciFig: Towards Automating Scientific Figure Generation

TL;DR

SciFig presents a multi-agent system that automatically generates publication-ready scientific pipeline figures from textual method descriptions. It combines hierarchical layout generation with an iterative chain-of-thought feedback loop and a rubric-based evaluation framework derived from 2{,}219 real figures, enabling objective quality assessment. Quantitative results show dataset-level quality of $70.1\%$ and paper-specific quality of $66.2\%$, with strong gains in structural coherence and visual clarity, while ablation studies confirm the necessity of both hierarchical layout and iterative feedback. In practice, SciFig delivers fully editable vector figures within minutes, achieving substantial speedups over manual figure creation and providing a scalable benchmark for future research in automatic scientific visualization.

Abstract

Creating high-quality figures and visualizations for scientific papers is a time-consuming task that requires both deep domain knowledge and professional design skills. Despite over 2.5 million scientific papers published annually, the figure generation process remains largely manual. We introduce $\textbf{SciFig}$, an end-to-end AI agent system that generates publication-ready pipeline figures directly from research paper texts. SciFig uses a hierarchical layout generation strategy, which parses research descriptions to identify component relationships, groups related elements into functional modules, and generates inter-module connections to establish visual organization. Furthermore, an iterative chain-of-thought (CoT) feedback mechanism progressively improves layouts through multiple rounds of visual analysis and reasoning. We introduce a rubric-based evaluation framework that analyzes 2,219 real scientific figures to extract evaluation rubrics and automatically generates comprehensive evaluation criteria. SciFig demonstrates remarkable performance: achieving 70.1$\%$ overall quality on dataset-level evaluation and 66.2$\%$ on paper-specific evaluation, and consistently high scores across metrics such as visual clarity, structural organization, and scientific accuracy. SciFig figure generation pipeline and our evaluation benchmark will be open-sourced.

Figures (9)

• Figure 1: SciFig: Automatically creates publication-ready figures. Existing methods either require days of manual work leblanc2019scientificjiang2019high or produce poor AI outputs zala2023diagrammergptchen2024makesshen2024sgneau2023fine (blurry images, wrong connections, flat layouts). SciFig is the first end-to-end system that automatically generates high-quality method figures through hierarchical layout generation, iterative improvement, and quality evaluation—reducing creation time from days to minutes while matching human-designed figures.
• Figure 2: SciFig Multi-Agent System Architecture. Our system consists of three stages: Hierarchical Layout Generation where the Description Agent parses research text, followed by the Layout Agent that organizes components into hierarchical structure. Iterative CoT Feedback where the Feedback Agent analyzes rendered layouts to identify specific issues, while the Layout Agent applies Chain-of-Thought reasoning to generate improved layouts across multiple rounds. Component Generation where the Component Agent renders individual visual elements with consistent styling to produce the final figure. This figure is drafted by our system and polished manually.
• Figure 3: Qualitative Comparison Across Research Domains. We compare SciFig against four baseline methods with same method text input across three diverse research domains: Artificial Intelligence (left column), Graphics (middle column), and Material Science (right column). GPT-5-Image produces figures with poor hierarchical organization, arbitrary arrow connections, inconsistent styling, and bad text rendering quality. Qwen-Image generates text-heavy outputs that completely fail to create visual pipeline representations, essentially producing unreadable document layouts instead of technical diagrams. Stable Diffusion XL produces blurry, non-editable images that lack scientific accuracy and semantic coherence. SciFig (Ours) generates publication-ready figures with clear hierarchical organization, clean module-level connections, consistent professional styling, and domain-appropriate scientific accuracy. Our method maintains high quality across diverse research domains, demonstrating the effectiveness of multi-agent architecture with hierarchical layout generation and iterative CoT feedback.
• Figure 4: Detailed Evaluation Rubrics. Our Evaluation Agent automatically derives six quality dimensions by analyzing 2,219 real scientific figures. Each rubric contains specific criteria that capture both visual properties (clarity, consistency, implementation) and semantic accuracy (technical correctness, structural coherence, interpretability). These rubrics form the foundation for our comprehensive figure evaluation framework, enabling systematic quality evaluation across diverse research domains.
• Figure 5: Evaluation Set Statistics. Distribution of 435 papers by conference (left) and research domain (right, top 20 shown). The dataset spans 15 top-tier AI conferences and 37 domains, with balanced representation across major research areas (cs.CV, cs.CL, cs.AI each with 41 papers).