Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization

Jinwei Lu, Yuanfeng Song, Chen Zhang, Raymond Chi-Wing Wong

TL;DR

This work tackles the reliability and versatility gap in automated multi-modal visualization by introducing MultiVis-Agent, a logic rule–enhanced multi-agent framework. It combines a centralized Coordinator with specialized agents for data querying, visualization code generation, and validation, guided by a four-layer framework of CR, TE, EH, and RC rules to provide formal guarantees on parameter safety, error recovery, and termination. The authors formalize the MultiVis task into four scenarios, create MultiVis-Bench with over 1,000 executable, multi-modal cases, and demonstrate substantial empirical gains in task completion and code execution across challenging tasks, outperforming baselines. The results indicate that constraint-guided LLM reasoning, coupled with robust coordination and cross-modal processing, yields a practical, scalable solution for reliable end-to-end visualization generation with iterative refinement.

Abstract

Real-world visualization tasks involve complex, multi-modal requirements that extend beyond simple text-to-chart generation, requiring reference images, code examples, and iterative refinement. Current systems exhibit fundamental limitations: single-modality input, one-shot generation, and rigid workflows. While LLM-based approaches show potential for these complex requirements, they introduce reliability challenges including catastrophic failures and infinite loop susceptibility. To address this gap, we propose MultiVis-Agent, a logic rule-enhanced multi-agent framework for reliable multi-modal and multi-scenario visualization generation. Our approach introduces a four-layer logic rule framework that provides mathematical guarantees for system reliability while maintaining flexibility. Unlike traditional rule-based systems, our logic rules are mathematical constraints that guide LLM reasoning rather than replacing it. We formalize the MultiVis task spanning four scenarios from basic generation to iterative refinement, and develop MultiVis-Bench, a benchmark with over 1,000 cases for multi-modal visualization evaluation. Extensive experiments demonstrate that our approach achieves 75.63% visualization score on challenging tasks, significantly outperforming baselines (57.54-62.79%), with task completion rates of 99.58% and code execution success rates of 94.56% (vs. 74.48% and 65.10% without logic rules), successfully addressing both complexity and reliability challenges in automated visualization generation.

MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization

TL;DR

This work tackles the reliability and versatility gap in automated multi-modal visualization by introducing MultiVis-Agent, a logic rule–enhanced multi-agent framework. It combines a centralized Coordinator with specialized agents for data querying, visualization code generation, and validation, guided by a four-layer framework of CR, TE, EH, and RC rules to provide formal guarantees on parameter safety, error recovery, and termination. The authors formalize the MultiVis task into four scenarios, create MultiVis-Bench with over 1,000 executable, multi-modal cases, and demonstrate substantial empirical gains in task completion and code execution across challenging tasks, outperforming baselines. The results indicate that constraint-guided LLM reasoning, coupled with robust coordination and cross-modal processing, yields a practical, scalable solution for reliable end-to-end visualization generation with iterative refinement.

Abstract

Real-world visualization tasks involve complex, multi-modal requirements that extend beyond simple text-to-chart generation, requiring reference images, code examples, and iterative refinement. Current systems exhibit fundamental limitations: single-modality input, one-shot generation, and rigid workflows. While LLM-based approaches show potential for these complex requirements, they introduce reliability challenges including catastrophic failures and infinite loop susceptibility. To address this gap, we propose MultiVis-Agent, a logic rule-enhanced multi-agent framework for reliable multi-modal and multi-scenario visualization generation. Our approach introduces a four-layer logic rule framework that provides mathematical guarantees for system reliability while maintaining flexibility. Unlike traditional rule-based systems, our logic rules are mathematical constraints that guide LLM reasoning rather than replacing it. We formalize the MultiVis task spanning four scenarios from basic generation to iterative refinement, and develop MultiVis-Bench, a benchmark with over 1,000 cases for multi-modal visualization evaluation. Extensive experiments demonstrate that our approach achieves 75.63% visualization score on challenging tasks, significantly outperforming baselines (57.54-62.79%), with task completion rates of 99.58% and code execution success rates of 94.56% (vs. 74.48% and 65.10% without logic rules), successfully addressing both complexity and reliability challenges in automated visualization generation.
Paper Structure (74 sections, 18 equations, 4 figures, 4 tables, 2 algorithms)

This paper contains 74 sections, 18 equations, 4 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Limitations.
  3. The Reliability Crisis.
  4. Our Solution: MultiVis-Agent.
  5. Key Innovations.
  6. Contributions.
  7. MultiVis: A Comprehensive Task for Visualization Generation
  8. Overview and Motivation
  9. Task Formulation
  10. MultiVis-Bench: Benchmark of MultiVis
  11. Overview and Motivation
  12. Dataset Construction Methodology
  13. Foundational Resources.
  14. Scenario-Specific Construction.
  15. Quality Assurance.
  16. ...and 59 more sections

Figures (4)

  • Figure 1: Real-world visualization tasks require multi-modal inputs and iterative refinement. Current Text-to-Vis systems fail to support these scenarios.
  • Figure 2: The architecture of MultiVis-Agent. The framework processes multi-modal user inputs (NLQ, Reference Chart, Reference Code, Iterative Code, Database) through a central Coordinator Agent that dynamically orchestrates specialized agents: the Database & Query Agent (DQ Agent), the Visualization Implementation Agent (Vis Agent), and the Validation & Evaluation Agent (VE Agent). Each agent is equipped with specific tools and follows a reasoning cycle (Thought-Observation-Action). The system is governed by a four-layer logic rule framework (CR, TE, EH, RC logic rules) that ensures reliable execution and error handling, enabling task-adaptive visualization generation and iterative refinement.
  • Figure 3: An example for the working process of MultiVis-Agent.
  • Figure 4: Qualitative comparison for an Image-Referenced Generation task. (a) shows the reference chart image provided by the user, (b) is the ground truth visualization, and (c) is the result generated by MultiVis-Agent that successfully matches both the reference style and user requirements. In contrast, both baseline methods - (d) LLM Workflow and (e) Instructing LLM - failed to generate visualizations that meet the user's expectations.