SentGraph: Hierarchical Sentence Graph for Multi-hop Retrieval-Augmented Question Answering

TL;DR

SentGraph tackles multi-hop retrieval-augmented QA by shifting from chunk-level to sentence-level reasoning and explicitly modeling sentence dependencies with an offline hierarchical graph built from an adapted RST framework. The method uses a three-layer graph (topic, core, supplementary) with nucleus-nucleus and nucleus-satellite relations, plus cross-document bridges, enabling fine-grained evidence selection and directed reasoning paths online. Through anchor refinement, adaptive path expansion, and evidence-based answer generation, SentGraph achieves state-of-the-art results on four benchmarks while reducing input/output token usage. This approach highlights the practical value of structured sentence-level reasoning for complex cross-document question answering and improves efficiency for real-world QA systems.

Abstract

Traditional Retrieval-Augmented Generation (RAG) effectively supports single-hop question answering with large language models but faces significant limitations in multi-hop question answering tasks, which require combining evidence from multiple documents. Existing chunk-based retrieval often provides irrelevant and logically incoherent context, leading to incomplete evidence chains and incorrect reasoning during answer generation. To address these challenges, we propose SentGraph, a sentence-level graph-based RAG framework that explicitly models fine-grained logical relationships between sentences for multi-hop question answering. Specifically, we construct a hierarchical sentence graph offline by first adapting Rhetorical Structure Theory to distinguish nucleus and satellite sentences, and then organizing them into topic-level subgraphs with cross-document entity bridges. During online retrieval, SentGraph performs graph-guided evidence selection and path expansion to retrieve fine-grained sentence-level evidence. Extensive experiments on four multi-hop question answering benchmarks demonstrate the effectiveness of SentGraph, validating the importance of explicitly modeling sentence-level logical dependencies for multi-hop reasoning.

Figures (8)

• Figure 1: Comparison of the traditional chunk-level and our adopted sentence-level graph construction methods.
• Figure 2: An overview of the SentGraph framework. The offline stage constructs a hierarchical sentence logic graph, and the online stage performs graph-based retrieval and answer generation for multi-hop question answering.
• Figure 3: Performance across multi-hop question answering datasets with varying anchor numbers.
• Figure 4: Efficiency analysis on average token usage per query across multi-hop question answering datasets.
• Figure 5: Prompt Template for N-N Relations Recognition.