Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FABLE: Forest-Based Adaptive Bi-Path LLM-Enhanced Retrieval for Multi-Document Reasoning

Lin Sun, Linglin Zhang, Jingang Huang, Change Jia, Zhengwei Cheng, Xiangzheng Zhang

TL;DR

FABLE addresses the inadequacies of long-context LLMs for multi-document reasoning by integrating LLM-driven knowledge organization with retrieval. It builds LLM-enhanced hierarchical forests as offline indexes and employs a budget-aware, bi-path traversal to balance coverage and detail, achieving high completeness with dramatically reduced token usage while maintaining faithfulness. Across synthetic and real-world benchmarks, FABLE outperforms state-of-the-art structured RAG systems and rivals full-context inference, illustrating that structured retrieval guided by cognitive-like navigation yields practical gains in efficiency and reliability. The framework demonstrates strong cross-domain applicability and highlights the importance of retrieval architecture in unlocking the potential of long-context LLMs for complex reasoning tasks.

Abstract

The rapid expansion of long-context Large Language Models (LLMs) has reignited debate on whether Retrieval-Augmented Generation (RAG) remains necessary. However, empirical evidence reveals persistent limitations of long-context inference, including the lost-in-the-middle phenomenon, high computational cost, and poor scalability for multi-document reasoning. Conversely, traditional RAG systems, while efficient, are constrained by flat chunk-level retrieval that introduces semantic noise and fails to support structured cross-document synthesis. We present \textbf{FABLE}, a \textbf{F}orest-based \textbf{A}daptive \textbf{B}i-path \textbf{L}LM-\textbf{E}nhanced retrieval framework that integrates LLMs into both knowledge organization and retrieval. FABLE constructs LLM-enhanced hierarchical forest indexes with multi-granularity semantic structures, then employs a bi-path strategy combining LLM-guided hierarchical traversal with structure-aware propagation for fine-grained evidence acquisition, with explicit budget control for adaptive efficiency trade-offs. Extensive experiments demonstrate that FABLE consistently outperforms SOTA RAG methods and achieves comparable accuracy to full-context LLM inference with up to 94\% token reduction, showing that long-context LLMs amplify rather than fully replace the need for structured retrieval.

FABLE: Forest-Based Adaptive Bi-Path LLM-Enhanced Retrieval for Multi-Document Reasoning

TL;DR

FABLE addresses the inadequacies of long-context LLMs for multi-document reasoning by integrating LLM-driven knowledge organization with retrieval. It builds LLM-enhanced hierarchical forests as offline indexes and employs a budget-aware, bi-path traversal to balance coverage and detail, achieving high completeness with dramatically reduced token usage while maintaining faithfulness. Across synthetic and real-world benchmarks, FABLE outperforms state-of-the-art structured RAG systems and rivals full-context inference, illustrating that structured retrieval guided by cognitive-like navigation yields practical gains in efficiency and reliability. The framework demonstrates strong cross-domain applicability and highlights the importance of retrieval architecture in unlocking the potential of long-context LLMs for complex reasoning tasks.

Abstract

The rapid expansion of long-context Large Language Models (LLMs) has reignited debate on whether Retrieval-Augmented Generation (RAG) remains necessary. However, empirical evidence reveals persistent limitations of long-context inference, including the lost-in-the-middle phenomenon, high computational cost, and poor scalability for multi-document reasoning. Conversely, traditional RAG systems, while efficient, are constrained by flat chunk-level retrieval that introduces semantic noise and fails to support structured cross-document synthesis. We present \textbf{FABLE}, a \textbf{F}orest-based \textbf{A}daptive \textbf{B}i-path \textbf{L}LM-\textbf{E}nhanced retrieval framework that integrates LLMs into both knowledge organization and retrieval. FABLE constructs LLM-enhanced hierarchical forest indexes with multi-granularity semantic structures, then employs a bi-path strategy combining LLM-guided hierarchical traversal with structure-aware propagation for fine-grained evidence acquisition, with explicit budget control for adaptive efficiency trade-offs. Extensive experiments demonstrate that FABLE consistently outperforms SOTA RAG methods and achieves comparable accuracy to full-context LLM inference with up to 94\% token reduction, showing that long-context LLMs amplify rather than fully replace the need for structured retrieval.
Paper Structure (22 sections, 8 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 22 sections, 8 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Long-Context Large Language Models
  4. Retrieval-Augmented Generation
  5. Structured Representations for RAG
  6. LLM-Augmented Retrieval and Reasoning
  7. Methodology
  8. LLM-Enhanced Hierarchical Forest
  9. Budget-Adaptive Bi-Path Retrieval
  10. Experimental Setup
  11. Evaluation Datasets
  12. Experimental Configuration
  13. Evaluation Metrics
  14. Evaluation
  15. Effectiveness on Multi-Document Reasoning
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of FABLE, a forest-based adaptive bi-path LLM-enhanced retrieval framework
  • Figure 2: Semantic tree construction and multi-granularity vector indexing: (1) semantic-aware chunking, (2) LLM-based hierarchical tree generation, (3) vector indexing of internal (toc+summary) and leaf (chunk) nodes for hierarchical retrieval.
  • Figure 3: Performance of retrieval and context construction strategies across varying input lengths. Long-context models use fixed 517K windows, while retrieval methods are tested from 1K to 128K tokens. Chunk-based retrieval shows limited gains and degrades at large contexts, whereas FABLE consistently improves completeness while reducing hallucination and irrelevance. FABLE (docs/llm-docs) achieves the best performance at moderate budgets.
  • Figure 4: bi-path performance across domains and languages.
  • Figure 5: bi-path performance across different query types.