Pruning Minimal Reasoning Graphs for Efficient Retrieval-Augmented Generation
Ning Wang, Kuanyan Zhu, Daniel Yuehwoon Yee, Yitang Gao, Shiying Huang, Zirun Xu, Sainyam Galhotra
TL;DR
The paper tackles inefficiencies in retrieval-augmented generation by shifting from passage-centric to structure-centric retrieval. AutoPrunedRetriever encodes knowledge as symbolic E–R–E triples stored in a compact codebook, builds local, coherent graph chunks, and retrieves reasoning paths via a coarse-to-fine process, followed by knowledge selection and compact, ID-based prompts. A two-layer entity consolidation and an adaptive DPO-based compression policy keep memory small while allowing long-running, evolving sessions and multi-agent pipelines; two front ends (REBEL and LLM extractor) demonstrate robustness. Empirically, the method achieves state-of-the-art complex reasoning on GraphRAG benchmarks and excels on STEM/TV datasets, delivering up to two orders of magnitude fewer tokens than graph-heavy baselines and enabling efficient, persistent reasoning in knowledge-intensive tasks.
Abstract
Retrieval-augmented generation (RAG) is now standard for knowledge-intensive LLM tasks, but most systems still treat every query as fresh, repeatedly re-retrieving long passages and re-reasoning from scratch, inflating tokens, latency, and cost. We present AutoPrunedRetriever, a graph-style RAG system that persists the minimal reasoning subgraph built for earlier questions and incrementally extends it for later ones. AutoPrunedRetriever stores entities and relations in a compact, ID-indexed codebook and represents questions, facts, and answers as edge sequences, enabling retrieval and prompting over symbolic structure instead of raw text. To keep the graph compact, we apply a two-layer consolidation policy (fast ANN/KNN alias detection plus selective $k$-means once a memory threshold is reached) and prune low-value structure, while prompts retain only overlap representatives and genuinely new evidence. We instantiate two front ends: AutoPrunedRetriever-REBEL, which uses REBEL as a triplet parser, and AutoPrunedRetriever-llm, which swaps in an LLM extractor. On GraphRAG-Benchmark (Medical and Novel), both variants achieve state-of-the-art complex reasoning accuracy, improving over HippoRAG2 by roughly 9--11 points, and remain competitive on contextual summarize and generation. On our harder STEM and TV benchmarks, AutoPrunedRetriever again ranks first, while using up to two orders of magnitude fewer tokens than graph-heavy baselines, making it a practical substrate for long-running sessions, evolving corpora, and multi-agent pipelines.