Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering

Yuling Shi, Maolin Sun, Zijun Liu, Mo Yang, Yixiong Fang, Tianran Sun, Xiaodong Gu

TL;DR

This work tackles multi-hop question answering with retrieval-augmented generation by introducing Reasoning Tree Guided RAG (RT-RAG), a hierarchical framework that forces explicit problem structure before retrieval. It decomposes questions into a reasoning tree using core queries, known and unknown entities, and then traverses the tree bottom-up, refining and rewriting queries while filtering inconsistent evidence. Key contributions include a consensus-based tree selection mechanism, adaptive leaf decomposition to avoid over-decomposition, rejection sampling to suppress hallucinations, and iterative refinement to ensure coherence across hops. Empirically, RT-RAG achieves state-of-the-art performance on three benchmarks, with average improvements of about 7.0% in F1 and 6.0% in EM, validating the approach as robust across model families and datasets.

Abstract

Retrieval-Augmented Generation (RAG) has demonstrated significant effectiveness in enhancing large language models (LLMs) for complex multi-hop question answering (QA). For multi-hop QA tasks, current iterative approaches predominantly rely on LLMs to self-guide and plan multi-step exploration paths during retrieval, leading to substantial challenges in maintaining reasoning coherence across steps from inaccurate query decomposition and error propagation. To address these issues, we introduce Reasoning Tree Guided RAG (RT-RAG), a novel hierarchical framework for complex multi-hop QA. RT-RAG systematically decomposes multi-hop questions into explicit reasoning trees, minimizing inaccurate decomposition through structured entity analysis and consensus-based tree selection that clearly separates core queries, known entities, and unknown entities. Subsequently, a bottom-up traversal strategy employs iterative query rewriting and refinement to collect high-quality evidence, thereby mitigating error propagation. Comprehensive experiments show that RT-RAG substantially outperforms state-of-the-art methods by 7.0% F1 and 6.0% EM, demonstrating the effectiveness of RT-RAG in complex multi-hop QA.

Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering

TL;DR

This work tackles multi-hop question answering with retrieval-augmented generation by introducing Reasoning Tree Guided RAG (RT-RAG), a hierarchical framework that forces explicit problem structure before retrieval. It decomposes questions into a reasoning tree using core queries, known and unknown entities, and then traverses the tree bottom-up, refining and rewriting queries while filtering inconsistent evidence. Key contributions include a consensus-based tree selection mechanism, adaptive leaf decomposition to avoid over-decomposition, rejection sampling to suppress hallucinations, and iterative refinement to ensure coherence across hops. Empirically, RT-RAG achieves state-of-the-art performance on three benchmarks, with average improvements of about 7.0% in F1 and 6.0% in EM, validating the approach as robust across model families and datasets.

Abstract

Retrieval-Augmented Generation (RAG) has demonstrated significant effectiveness in enhancing large language models (LLMs) for complex multi-hop question answering (QA). For multi-hop QA tasks, current iterative approaches predominantly rely on LLMs to self-guide and plan multi-step exploration paths during retrieval, leading to substantial challenges in maintaining reasoning coherence across steps from inaccurate query decomposition and error propagation. To address these issues, we introduce Reasoning Tree Guided RAG (RT-RAG), a novel hierarchical framework for complex multi-hop QA. RT-RAG systematically decomposes multi-hop questions into explicit reasoning trees, minimizing inaccurate decomposition through structured entity analysis and consensus-based tree selection that clearly separates core queries, known entities, and unknown entities. Subsequently, a bottom-up traversal strategy employs iterative query rewriting and refinement to collect high-quality evidence, thereby mitigating error propagation. Comprehensive experiments show that RT-RAG substantially outperforms state-of-the-art methods by 7.0% F1 and 6.0% EM, demonstrating the effectiveness of RT-RAG in complex multi-hop QA.
Paper Structure (19 sections, 3 figures, 6 tables)

This paper contains 19 sections, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval Augmented Generation
  4. Multi-hop Question Answering
  5. Methodology
  6. Question Decomposition
  7. Retrieval and Answer Aggregation
  8. Query Rewriting
  9. Answer Integration and Iterative Refinement
  10. Experiments
  11. Datasets
  12. Baselines
  13. Metrics
  14. Experimental Setup
  15. Results and Analysis
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Examples of two common challenges in multi-hop QA: error propagation and inaccurate query decomposition.
  • Figure 2: Overview of the RT-RAG framework. RT-RAG first decomposes the complex question into a consensus-validated tree structure with explicit entity analysis, then retrieves evidence through bottom-up traversal with query refinement, and finally integrates information hierarchically to maintain coherence across multiple hops.
  • Figure 3: Tree depth distribution on different datasets with Qwen2.5-14B-Instruct model.