Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA

Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang, Jiahao Gu, Zimeng Chen, Menglan Chen, Wen Yao, Yang Yang, Ying Wen, Siheng Chen

TL;DR

G^2-Reader introduces a dual-graph framework for multimodal long-document QA, coupling a Content Graph that preserves document-native structure and cross-modal relations with a Planning Graph that explicitly models sub-question reasoning. The Content Graph is iteratively evolved via a vision-language model to fuse heterogeneous evidence, while the Planning Graph orchestrates stepwise retrieval and verification, updating plans based on evidence gaps. On VisDoMBench across five domains, the approach with an open-source backbone achieves 66.21% average accuracy, outperforming strong baselines and a stand-alone GPT-5 (53.08%). Ablation studies show complementary benefits from both graphs and demonstrate practical efficiency gains with a lite variant that replaces VLM-driven evolution with a lightweight embedding-based update.

Abstract

Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal alignment, yielding semantic fragments that are hard to interpret in isolation. Second, even iterative retrieval can fail in long contexts by looping on partial evidence or drifting into irrelevant sections as noise accumulates, since each step is guided only by the current snippet without a persistent global search state. We introduce $G^2$-Reader, a dual-graph system, to address both issues. It evolves a Content Graph to preserve document-native structure and cross-modal semantics, and maintains a Planning Graph, an agentic directed acyclic graph of sub-questions, to track intermediate findings and guide stepwise navigation for evidence completion. On VisDoMBench across five multimodal domains, $G^2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21\% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08\%).

$G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA

TL;DR

G^2-Reader introduces a dual-graph framework for multimodal long-document QA, coupling a Content Graph that preserves document-native structure and cross-modal relations with a Planning Graph that explicitly models sub-question reasoning. The Content Graph is iteratively evolved via a vision-language model to fuse heterogeneous evidence, while the Planning Graph orchestrates stepwise retrieval and verification, updating plans based on evidence gaps. On VisDoMBench across five domains, the approach with an open-source backbone achieves 66.21% average accuracy, outperforming strong baselines and a stand-alone GPT-5 (53.08%). Ablation studies show complementary benefits from both graphs and demonstrate practical efficiency gains with a lite variant that replaces VLM-driven evolution with a lightweight embedding-based update.

Abstract

Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal alignment, yielding semantic fragments that are hard to interpret in isolation. Second, even iterative retrieval can fail in long contexts by looping on partial evidence or drifting into irrelevant sections as noise accumulates, since each step is guided only by the current snippet without a persistent global search state. We introduce -Reader, a dual-graph system, to address both issues. It evolves a Content Graph to preserve document-native structure and cross-modal semantics, and maintains a Planning Graph, an agentic directed acyclic graph of sub-questions, to track intermediate findings and guide stepwise navigation for evidence completion. On VisDoMBench across five multimodal domains, -Reader with Qwen3-VL-32B-Instruct reaches 66.21\% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08\%).
Paper Structure (38 sections, 11 equations, 10 figures, 5 tables)

This paper contains 38 sections, 11 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Memory Construction for RAG
  4. Agentic Retrieval with Reasoning
  5. Methodology: $G^2$-Reader
  6. Problem Formulation
  7. Content Graph
  8. Automated Graph Construction
  9. Joint Graph Evolution of Topology and Node Attributes
  10. Structured Subgraph Readout
  11. Dynamic Planning Graph Evolution
  12. Initial Planning Graph Construction
  13. Planning Graph Execution and Refinement
  14. Evidence-Grounded Answer Synthesis
  15. Experiments
  16. ...and 23 more sections

Figures (10)

  • Figure 1: Overview of the $G^2$-Reader framework (Left). $G^2$-Reader bridges the gap between offline document indexing and online inference through a dual-graph architecture: (Right) The Content Graph ($\mathcal{G}_C$) provides a structured multimodal representation of documents with evolved node attributes; (Middle) The Planning Graph ($\mathcal{G}_P$) serves as an explicit, dynamic reasoning state that iteratively orchestrates sub-question decomposition and evidence assembly.
  • Figure 2: Ablation on $k$.
  • Figure 3: Ablation on model size.
  • Figure 5: Multimodal Content Graph of a representative SciGraphQA example based on panda2017unsupervised. Node summaries are further manually condensed and annotated for readability.
  • Figure 6: Example of Planning Graph Updating. Nodes updates and additions to bridge information gaps.
  • ...and 5 more figures