Doc-Researcher: A Unified System for Multimodal Document Parsing and Deep Research

TL;DR

Doc-Researcher tackles the limitation of existing deep research systems that rely only on textual data by enabling multimodal document understanding across text, tables, figures, and equations. It introduces a deep multimodal parsing framework that preserves layout and visuals, a retrieval architecture supporting text, vision, and hybrid modalities with dynamic granularity, and an iterative multi-agent deep research workflow for progressive evidence synthesis. To evaluate the approach, the authors propose M4DocBench, a benchmark featuring 158 expert-annotated questions over 304 documents across four domains, enabling multi-hop, multi-modal, multi-document, and multi-turn assessment. Experiments show 50.6% answer accuracy and a 3.4× improvement over baselines, validating the importance of deep parsing and adaptive iterative reasoning for complex multimodal document research.

Abstract

Deep Research systems have revolutionized how LLMs solve complex questions through iterative reasoning and evidence gathering. However, current systems remain fundamentally constrained to textual web data, overlooking the vast knowledge embedded in multimodal documents Processing such documents demands sophisticated parsing to preserve visual semantics (figures, tables, charts, and equations), intelligent chunking to maintain structural coherence, and adaptive retrieval across modalities, which are capabilities absent in existing systems. In response, we present Doc-Researcher, a unified system that bridges this gap through three integrated components: (i) deep multimodal parsing that preserves layout structure and visual semantics while creating multi-granular representations from chunk to document level, (ii) systematic retrieval architecture supporting text-only, vision-only, and hybrid paradigms with dynamic granularity selection, and (iii) iterative multi-agent workflows that decompose complex queries, progressively accumulate evidence, and synthesize comprehensive answers across documents and modalities. To enable rigorous evaluation, we introduce M4DocBench, the first benchmark for Multi-modal, Multi-hop, Multi-document, and Multi-turn deep research. Featuring 158 expert-annotated questions with complete evidence chains across 304 documents, M4DocBench tests capabilities that existing benchmarks cannot assess. Experiments demonstrate that Doc-Researcher achieves 50.6% accuracy, 3.4xbetter than state-of-the-art baselines, validating that effective document research requires not just better retrieval, but fundamentally deep parsing that preserve multimodal integrity and support iterative research. Our work establishes a new paradigm for conducting deep research on multimodal document collections.

Figures (7)

• Figure 1: An typical use case of multimodal doc deep research. The user asks a multi-hop question in the context of multi-turn conversations, where ground-truth evidence spans across multiple documents and modalities. The demo conversations, evidence, and answer are shortened for simplicity. Refer to more examples and evidence-chain annotations in Figure \ref{['fig:annotations']}.
• Figure 2: Doc-Researcher Architecture: (a) multimodal deep parsing and indexing, and (b) multimodal deep research.
• Figure 3: Performance with increasing search depth.
• Figure 4: Demonstration of two typical annotations of M4DocBench. Note that they are simplified for brevity.
• Figure 5: The overview of layout-aware chunking process and multimodal-to-text transcription.