StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

TL;DR

This work tackles the challenge of knowledge-intensive reasoning in LLMs by introducing StructRAG, a framework that selectively applies the most suitable knowledge structure type for a given task. It integrates a DPO-trained hybrid structure router, an LLM-based scattered knowledge structurizer, and a structured knowledge utilizer to decompose problems and extract precise information from structured representations. Empirical results on the Loong benchmark and Podcast Transcripts show state-of-the-art performance, particularly as information becomes more dispersed, and the approach remains faster than GraphRAG baselines. Overall, StructRAG demonstrates the value of cognitive-inspired, hybrid knowledge structuring for enhancing reasoning in knowledge-intensive NLP tasks and points to strong practical potential for real-world applications.

Abstract

Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks. However, existing RAG methods struggle with knowledge-intensive reasoning tasks, because useful information required to these tasks are badly scattered. This characteristic makes it difficult for existing RAG methods to accurately identify key information and perform global reasoning with such noisy augmentation. In this paper, motivated by the cognitive theories that humans convert raw information into various structured knowledge when tackling knowledge-intensive reasoning, we proposes a new framework, StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Extensive experiments across various knowledge-intensive tasks show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios, demonstrating its potential as an effective solution for enhancing LLMs in complex real-world applications.

Figures (9)

• Figure 1: The overview of StructRAG framework, including an hybrid structure router to select the optimal structure type based on task requirements, a scattered knowledge structurizer to convert raw documents into structured knowledge, and a structured knowledge utilizer to decompose complex question and then effectively using the structured knowledge to infer the final answer.
• Figure 2: The illustration of training data constructing. First use LLMs to synthesize knowledge-intensive tasks, and then simulate solutions by structured knowledge in different types, finally judge all possible solutions and get preference pairs about candidate structure types.
• Figure 3: Performance of StructRAG with different routers. The strong router shows obvious positive impact on the overall framework.
• Figure 4: Results of evaluating hybrid structure routers. The table shows that preference training is necessary for the routing ability.
• Figure 5: Comparison of implementing latency (minute). The StructRAG has an available speed, which is a little slower than RQ-RAG, but is much faster than GraphRAG method.