Cognitive Graph for Multi-Hop Reading Comprehension at Scale

TL;DR

CogQA tackles open-domain, multi-hop reading comprehension by modeling reasoning as an explicit cognitive graph built through a dual-system process. System 1 (BERT-based extraction) incrementally gathers candidate hops and answers guided by clues, while System 2 (a relational GNN) performs iterative relational reasoning over the graph to refine representations and guide further extraction. The predictor uses node embeddings to produce final answers, trained with negative sampling across two tasks: span extraction and answer-node prediction. On HotpotQA fullwiki, CogQA achieves state-of-the-art joint metrics (notably a joint $F_1$ of $34.9$), demonstrating strong scalability to web-scale document collections and providing clear, entity-level explanation paths for multi-hop reasoning.

Abstract

We propose a new CogQA framework for multi-hop question answering in web-scale documents. Inspired by the dual process theory in cognitive science, the framework gradually builds a \textit{cognitive graph} in an iterative process by coordinating an implicit extraction module (System 1) and an explicit reasoning module (System 2). While giving accurate answers, our framework further provides explainable reasoning paths. Specifically, our implementation based on BERT and graph neural network efficiently handles millions of documents for multi-hop reasoning questions in the HotpotQA fullwiki dataset, achieving a winning joint $F_1$ score of 34.9 on the leaderboard, compared to 23.6 of the best competitor.

Figures (4)

• Figure 1: An example of cognitive graph for multi-hop QA. Each hop node corresponds to an entity (e.g., "Los Angeles") followed by its introductory paragraph. The circles mean ans nodes, answer candidates to the question. Cognitive graph mimics human reasoning process. Edges are built when calling an entity to "mind". The solid black edges are the correct reasoning path.
• Figure 2: Overview of CogQA implementation. When visiting the node $x$, System 1 generates new hop and answer nodes based on the $clues[x,\mathcal{G}]$ discovered by System 2. It also creates the inital representation $sem[x,Q,clues]$, based on which the GNN in System 2 updates the hidden representations $\mathbf{X}[x]$.
• Figure 3: Model performance on 8 types of questions with different hops.
• Figure 4: Case Study. Different forms of cognitive graphs in our results, i.e., Tree, Directed Acyclic Graph (DAG), Cyclic Graph. Circles are candidate answer nodes while rounded rectangles are hop nodes. Green circles are the final answers given by CogQA and check marks represent the annotated ground truth.