MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAG

TL;DR

MacRAG tackles the long-context RAG challenge by coupling offline hierarchical document indexing with a bottom-up, adaptive retrieval that starts from fine-grained slices and expands to larger chunks and documents to form query-specific long contexts. It achieves this via three phases: (i) top-down offline indexing (chunking, compression, slicing) to build a multi-scale hierarchy, (ii) bottom-up real-time retrieval and ranking (slice-level retrieval, chunk re-ranking, scaling-up, and neighbor propagation) to assemble bounded long contexts, and (iii) modular generation that consumes the assembled context. Empirical results on LongBench across HotpotQA, 2WikiMultihopQA, and MuSiQue with Llama-3.1-8B, Gemini-1.5-pro, and GPT-4o show MacRAG outperforms RAPTOR and LongRAG in single- and multi-step generation, with robust gains across generation modes and stronger improvements for larger models. The framework delivers efficient, scalable long-context RAG, maintains bounded context sizes, and reduces evidence fragmentation, making it well-suited for real-world multi-hop reasoning and enterprise-scale retrieval tasks; code is available at the provided GitHub repository.

Abstract

Long-context large language models (LC LLMs) combined with retrieval-augmented generation (RAG) hold strong potential for complex multi-hop and large-document tasks. However, existing RAG systems often suffer from imprecise retrieval, incomplete context coverage under constrained windows, and fragmented information from suboptimal context construction. We introduce Multi-scale Adaptive Context RAG (MacRAG), a hierarchical RAG framework that compresses and partitions documents into coarse-to-fine granularities, then adaptively merges relevant contexts through real-time chunk- and document-level expansions. By initiating with finest-level retrieval and progressively incorporating broader, higher-level context, MacRAG constructs effective query-specific long contexts, optimizing both precision and coverage. Evaluations on challenging LongBench expansions of HotpotQA, 2WikiMultihopQA, and Musique confirm MacRAG consistently surpasses baseline RAG pipelines in single- and multi-step generation using Llama-3.1-8B, Gemini-1.5-pro, and GPT-4o. Our results establish MacRAG as an efficient, scalable solution for real-world long-context, multi-hop reasoning. Our code is available at https://github.com/Leezekun/MacRAG.

Figures (5)

• Figure 1: An overview of the MacRAG framework, consisting of two main phases: (1) top-down hierarchical indexing (upper), and (2) bottom-up multi-scale adaptive retrieval on the constructed hierarchy of document-chunk-summary-slice (lower).
• Figure 2: Performance comparison of GPT-4o using LongRAG and MacRAG across the seven settings in Table \ref{['tbl_generation_modes']}, showing F1-scores for three datasets (HotpotQA, 2WikiMultihopQA, and MuSiQue). Complete results for all metrics and LLMs are provided in the Appendix.
• Figure 3: Performance trends across datasets for scale factors ($\alpha$).
• Figure 4: Cumulative LLM's input context lengths for single/multi-step generation methods in Table \ref{['tbl_generation_modes']}.
• Figure 5: Performances of LongRAG and MacRAG regarding the fours metrics (Exact Match, F1-score, Precision, Recall) for three datasets (HotpotQA, 2WikimultihopQA, and Musique) and two LLMs (Gemini-1.5-pro and GPT-4o). Each row corresponds to a combination of dataset and LLM, and each column represents one of the metrics.