RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation

TL;DR

RichRAG addresses the challenge of open-ended, multi-aspect queries by explicitly predicting sub-aspects, assembling a diverse external document pool, and ranking it with a generative list-wise model that captures global interactions and aligns with LLM preferences. The learning process combines supervised fine-tuning with a coverage-based silver target and reinforcement learning via Direct Preference Optimization, augmented by unilateral sample significance to stabilize training. Empirical results on WikiPassageQA and WikiAsp demonstrate improved coverage and more satisfying long-form responses compared with baselines, along with favorable efficiency characteristics. Overall, RichRAG advances retrieval-augmented generation by enabling richer, more reliable responses to complex user intents in open-domain QA.

Abstract

Retrieval-augmented generation (RAG) effectively addresses issues of static knowledge and hallucination in large language models. Existing studies mostly focus on question scenarios with clear user intents and concise answers. However, it is prevalent that users issue broad, open-ended queries with diverse sub-intents, for which they desire rich and long-form answers covering multiple relevant aspects. To tackle this important yet underexplored problem, we propose a novel RAG framework, namely RichRAG. It includes a sub-aspect explorer to identify potential sub-aspects of input questions, a multi-faceted retriever to build a candidate pool of diverse external documents related to these sub-aspects, and a generative list-wise ranker, which is a key module to provide the top-k most valuable documents for the final generator. These ranked documents sufficiently cover various query aspects and are aware of the generator's preferences, hence incentivizing it to produce rich and comprehensive responses for users. The training of our ranker involves a supervised fine-tuning stage to ensure the basic coverage of documents, and a reinforcement learning stage to align downstream LLM's preferences to the ranking of documents. Experimental results on two publicly available datasets prove that our framework effectively and efficiently provides comprehensive and satisfying responses to users.

Figures (8)

• Figure 1: An example of a scenario where a multi-faceted query requires a comprehensive answer.
• Figure 2: The overall framework of RichRAG. We describe the training stages of our ranker at the bottom.
• Figure 3: Results of the GPT-4-based evaluation comparing our method with baseline models.
• Figure 4: Ablation results of RichRAG on two datasets.
• Figure 5: Efficiency experiments of different models.