Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs

Ming Tu, Guangtao Wang, Jing Huang, Yun Tang, Xiaodong He, Bowen Zhou

TL;DR

This work tackles multi-hop reading comprehension across multiple documents by introducing the Heterogeneous Document-Entity (HDE) graph, which jointly represents documents, candidate answers, and entity mentions as distinct node types. By combining context-encoding with co-attention and self-attention and performing GNN-based message passing over multiple edge types, the model effectively propagates evidence across documents. The approach achieves competitive results in a single model and state-of-the-art performance when ensembled on WikiHop, highlighting the value of heterogeneous graphs for cross-document reasoning. The study also includes thorough ablations showing the importance of the heterogeneous graph and edge-type-aware message passing, with insights into how multi-hop reasoning benefits from across-document interactions.

Abstract

Multi-hop reading comprehension (RC) across documents poses new challenge over single-document RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to tackle the multi-hop RC problem. We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph. The advantage of HDE graph is that it contains different granularity levels of information including candidates, documents and entities in specific document contexts. Our proposed model can do reasoning over the HDE graph with nodes representation initialized with co-attention and self-attention based context encoders. We employ Graph Neural Networks (GNN) based message passing algorithms to accumulate evidences on the proposed HDE graph. Evaluated on the blind test set of the Qangaroo WikiHop data set, our HDE graph based single model delivers competitive result, and the ensemble model achieves the state-of-the-art performance.

Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs

TL;DR

This work tackles multi-hop reading comprehension across multiple documents by introducing the Heterogeneous Document-Entity (HDE) graph, which jointly represents documents, candidate answers, and entity mentions as distinct node types. By combining context-encoding with co-attention and self-attention and performing GNN-based message passing over multiple edge types, the model effectively propagates evidence across documents. The approach achieves competitive results in a single model and state-of-the-art performance when ensembled on WikiHop, highlighting the value of heterogeneous graphs for cross-document reasoning. The study also includes thorough ablations showing the importance of the heterogeneous graph and edge-type-aware message passing, with insights into how multi-hop reasoning benefits from across-document interactions.

Abstract

Multi-hop reading comprehension (RC) across documents poses new challenge over single-document RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to tackle the multi-hop RC problem. We introduce a heterogeneous graph with different types of nodes and edges, which is named as Heterogeneous Document-Entity (HDE) graph. The advantage of HDE graph is that it contains different granularity levels of information including candidates, documents and entities in specific document contexts. Our proposed model can do reasoning over the HDE graph with nodes representation initialized with co-attention and self-attention based context encoders. We employ Graph Neural Networks (GNN) based message passing algorithms to accumulate evidences on the proposed HDE graph. Evaluated on the blind test set of the Qangaroo WikiHop data set, our HDE graph based single model delivers competitive result, and the ensemble model achieves the state-of-the-art performance.

Paper Structure

This paper contains 14 sections, 12 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Context encoding
  5. Reasoning over HDE graph
  6. Score accumulation
  7. Experiments
  8. Dataset
  9. Experimental settings
  10. Results
  11. Ablation studies
  12. Result analysis
  13. Conclusion
  14. Acknowledgements

Figures (5)

  • Figure 1: A WikiHop example. Words with different colors indicate the evidences across documents.
  • Figure 2: System diagram. $S$ and $C$ are the number of support documents and candidates respectively. We use yellow nodes to represent query-aware candidate representation, blue nodes to represent extracted query-aware entity representation and green nodes to represent query-aware document representation.
  • Figure 3: A toy example of HDE graph. The dash dot lines connecting documents (green nodes) and candidates (yellow nodes) correspond to type 1 edge. The normal dash lines connecting documents and entities (blue nodes) correspond to type 2 edge. The square dot lines connecting entities and candidates correspond to type 3 edge. The red solid line connecting two entities correspond to type 4 edge. The purple solid line correspond to type 5 edge. The black solid lines connecting two candidates correspond to type 6 edge. For good visualization, we ignore the type 7 edge in this figure.
  • Figure 4: Plots between number of support documents (x-axis) and number of examples (left y-axis), and between number of support documents and accuracy (right y-axis).
  • Figure 5: Plots between number of candidates (x-axis) and number of examples (left y-axis), and between number of candidates and accuracy (right y-axis).