Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence

Biao Xiang, Soyeon Caren Han, Yihao Ding

TL;DR

BRIDGE is introduced, a benchmark for multi-hop reasoning over long scientific papers that require integrating evidence across text, tables, and figures, and provides explicit multi-hop reasoning annotations for step-level evaluation beyond answer accuracy.

Abstract

Multi-hop question answering (QA) is widely used to evaluate the reasoning capabilities of large language models, yet most benchmarks focus on final answer correctness and overlook intermediate reasoning, especially in long multimodal documents. We introduce BRIDGE, a benchmark for multi-hop reasoning over long scientific papers that require integrating evidence across text, tables, and figures. The dataset supports both chain-like and fan-out structures and provides explicit multi-hop reasoning annotations for step-level evaluation beyond answer accuracy. Experiments with state-of-the-art LLMs and multimodal retrieval-augmented generation (RAG) systems reveal systematic deficiencies in evidence aggregation and grounding that remain hidden under conventional answer-only evaluation. BRIDGE provides a targeted testbed for diagnosing reasoning failures in long multimodal documents.

BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence

TL;DR

BRIDGE is introduced, a benchmark for multi-hop reasoning over long scientific papers that require integrating evidence across text, tables, and figures, and provides explicit multi-hop reasoning annotations for step-level evaluation beyond answer accuracy.

Abstract

Multi-hop question answering (QA) is widely used to evaluate the reasoning capabilities of large language models, yet most benchmarks focus on final answer correctness and overlook intermediate reasoning, especially in long multimodal documents. We introduce BRIDGE, a benchmark for multi-hop reasoning over long scientific papers that require integrating evidence across text, tables, and figures. The dataset supports both chain-like and fan-out structures and provides explicit multi-hop reasoning annotations for step-level evaluation beyond answer accuracy. Experiments with state-of-the-art LLMs and multimodal retrieval-augmented generation (RAG) systems reveal systematic deficiencies in evidence aggregation and grounding that remain hidden under conventional answer-only evaluation. BRIDGE provides a targeted testbed for diagnosing reasoning failures in long multimodal documents.
Paper Structure (14 sections, 2 figures, 7 tables)

This paper contains 14 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Dataset
  3. QA Pair Generation
  4. Dataset Analysis
  5. Experimental Setup
  6. Results and Discussion
  7. Overall Performance
  8. Performance on Various Evaluation Metrics
  9. Breakdown Analysis
  10. Question-Type Analysis
  11. Cross-Page Evidence Distribution Analysis
  12. Hop Depth Analysis
  13. Diverse Evidence Modality Analysis
  14. Conclusion

Figures (2)

  • Figure 1: Representative examples of comparative (Cp), abstractive (Ab), and causal reasoning (Re) questions (top), and the corresponding pages where evidences locate (bottom). Mod.: involved modalities (T: text; Tb: table; F: figure).
  • Figure 2: Distribution of QA instances by hop depth, number of distinct pages involved, and hop pattern, broken down by question type (Abstractive, Causal, Comparative)