Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysis

Ruyi Qi, Zhou Liu, Wentao Zhang

TL;DR

DataCross addresses the real world need to activate and reason over cross modal heterogeneous data by introducing DataCrossBench, a 200 task benchmark with ground truth derived through a human in the loop reverse synthesis pipeline, and DataCrossAgent, a collaborative agent framework that orchestrates specialized sub agents and reReasoning to perform cross modal extraction, alignment, and joint reasoning. The framework uses a recursive reReasoning Act loop and a hybrid priority scoring scheme to identify key data sources and perform semantic cross pollination while a dedicated cross analysis component executes numeric joins to prevent hallucinations. Experimental results show DataCrossAgent achieving superior Four dim insight scores, notably a 29.7% improvement in factuality over GPT-4o and strong robustness on hard tasks, demonstrating improved activation of zombie visual data for grounded insights. The work provides a practical, end to end solution for industrial grade heterogeneous data analysis with verifiable ground truth and traceable evidence linking across structured and visual sources.

Abstract

In real-world data science and enterprise decision-making, critical information is often fragmented across directly queryable structured sources (e.g., SQL, CSV) and "zombie data" locked in unstructured visual documents (e.g., scanned reports, invoice images). Existing data analytics agents are predominantly limited to processing structured data, failing to activate and correlate this high-value visual information, thus creating a significant gap with industrial needs. To bridge this gap, we introduce DataCross, a novel benchmark and collaborative agent framework for unified, insight-driven analysis across heterogeneous data modalities. DataCrossBench comprises 200 end-to-end analysis tasks across finance, healthcare, and other domains. It is constructed via a human-in-the-loop reverse-synthesis pipeline, ensuring realistic complexity, cross-source dependency, and verifiable ground truth. The benchmark categorizes tasks into three difficulty tiers to evaluate agents' capabilities in visual table extraction, cross-modal alignment, and multi-step joint reasoning. We also propose the DataCrossAgent framework, inspired by the "divide-and-conquer" workflow of human analysts. It employs specialized sub-agents, each an expert on a specific data source, which are coordinated via a structured workflow of Intra-source Deep Exploration, Key Source Identification, and Contextual Cross-pollination. A novel reReAct mechanism enables robust code generation and debugging for factual verification. Experimental results show that DataCrossAgent achieves a 29.7% improvement in factuality over GPT-4o and exhibits superior robustness on high-difficulty tasks, effectively activating fragmented "zombie data" for insightful, cross-modal analysis.

DataCross: A Unified Benchmark and Agent Framework for Cross-Modal Heterogeneous Data Analysis

TL;DR

DataCross addresses the real world need to activate and reason over cross modal heterogeneous data by introducing DataCrossBench, a 200 task benchmark with ground truth derived through a human in the loop reverse synthesis pipeline, and DataCrossAgent, a collaborative agent framework that orchestrates specialized sub agents and reReasoning to perform cross modal extraction, alignment, and joint reasoning. The framework uses a recursive reReasoning Act loop and a hybrid priority scoring scheme to identify key data sources and perform semantic cross pollination while a dedicated cross analysis component executes numeric joins to prevent hallucinations. Experimental results show DataCrossAgent achieving superior Four dim insight scores, notably a 29.7% improvement in factuality over GPT-4o and strong robustness on hard tasks, demonstrating improved activation of zombie visual data for grounded insights. The work provides a practical, end to end solution for industrial grade heterogeneous data analysis with verifiable ground truth and traceable evidence linking across structured and visual sources.

Abstract

In real-world data science and enterprise decision-making, critical information is often fragmented across directly queryable structured sources (e.g., SQL, CSV) and "zombie data" locked in unstructured visual documents (e.g., scanned reports, invoice images). Existing data analytics agents are predominantly limited to processing structured data, failing to activate and correlate this high-value visual information, thus creating a significant gap with industrial needs. To bridge this gap, we introduce DataCross, a novel benchmark and collaborative agent framework for unified, insight-driven analysis across heterogeneous data modalities. DataCrossBench comprises 200 end-to-end analysis tasks across finance, healthcare, and other domains. It is constructed via a human-in-the-loop reverse-synthesis pipeline, ensuring realistic complexity, cross-source dependency, and verifiable ground truth. The benchmark categorizes tasks into three difficulty tiers to evaluate agents' capabilities in visual table extraction, cross-modal alignment, and multi-step joint reasoning. We also propose the DataCrossAgent framework, inspired by the "divide-and-conquer" workflow of human analysts. It employs specialized sub-agents, each an expert on a specific data source, which are coordinated via a structured workflow of Intra-source Deep Exploration, Key Source Identification, and Contextual Cross-pollination. A novel reReAct mechanism enables robust code generation and debugging for factual verification. Experimental results show that DataCrossAgent achieves a 29.7% improvement in factuality over GPT-4o and exhibits superior robustness on high-difficulty tasks, effectively activating fragmented "zombie data" for insightful, cross-modal analysis.
Paper Structure (55 sections, 7 equations, 5 figures, 3 tables)

This paper contains 55 sections, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Insight Discovery and Analytics
  4. Data Analysis Agents
  5. DataCrossBench: A Multi-Source Heterogeneous Benchmark
  6. Design Philosophy and Comparison
  7. Data Construction
  8. Dataset Statistics and Diversity
  9. Quality Control and Human Verification
  10. Evaluation Metrics
  11. Factuality ($S_{fact}$)
  12. Completeness ($S_{comp}$)
  13. Logic ($S_{logic}$)
  14. Insightfulness ($S_{ins}$)
  15. DataCrossAgent: A Corresponding Agent Framework
  16. ...and 40 more sections

Figures (5)

  • Figure 1: DataCrossBench distributions. (a) Domain distribution. (b) Modality distribution. (c) Difficulty tier breakdown.
  • Figure 2: The overall architecture of the DataCross framework. A multi-agent orchestration to integrate heterogeneous data sources.
  • Figure 3: Performance trends across different ablation and different levels.
  • Figure 4: The AI-Assisted Data Generator Platform. This interface facilitates the "Reverse-Synthesis" pipeline, allowing experts to produce large-scale, heterogeneous datasets that adhere to specific logical constraints.
  • Figure 5: The DataCross Verification Platform. This tool is used for the final quality audit, ensuring that the synthesized heterogeneous data supports the intended analysis goals without logical flaws or artifacts.