Searching for Best Practices in Retrieval-Augmented Generation

TL;DR

This work systematically investigates best practices for retrieval-augmented generation (RAG) by dissecting the full pipeline into modular components and evaluating representative methods across them. It introduces a structured, three-step experimental approach to select per-module choices and their combinations, and validates these findings through extensive benchmarks spanning open-domain, multi-hop, and medical QA tasks, alongside a multimodal extension. Key contributions include a comprehensive evaluation framework, performance-efficiency trade-offs for retrieval, reranking, summarization, and generator fine-tuning, and practical deployment recipes for high performance or balanced efficiency. The results offer actionable guidance for building scalable, low-latency RAG systems and demonstrate the added value of multimodal retrieval in grounding and accelerating multimodal content generation.

Abstract

Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, particularly in specialized domains. While many RAG approaches have been proposed to enhance large language models through query-dependent retrievals, these approaches still suffer from their complex implementation and prolonged response times. Typically, a RAG workflow involves multiple processing steps, each of which can be executed in various ways. Here, we investigate existing RAG approaches and their potential combinations to identify optimal RAG practices. Through extensive experiments, we suggest several strategies for deploying RAG that balance both performance and efficiency. Moreover, we demonstrate that multimodal retrieval techniques can significantly enhance question-answering capabilities about visual inputs and accelerate the generation of multimodal content using a "retrieval as generation" strategy.

Figures (4)

• Figure 1: Retrieval-augmented generation workflow. This study investigates the contribution of each component and provides insights into optimal RAG practices through extensive experimentation. The optional methods considered for each component are indicated in bold fonts, while the methods underlined indicate the default choice for individual modules. The methods indicated in blue font denote the best-performing selections identified empirically.
• Figure 2: Classification of retrieval requirements for different tasks. In cases where information is not provided, we differentiate tasks based on the functions of the model.
• Figure 3: Results of generator fine-tuning.
• Figure 4: Workflow of multimodal retrieval. The upper section illustrates the text-to-image retrieval process. Initially, a text query is used to find images in the database with the highest similarity. If a high similarity is found, the image is returned directly. If not, an image generation model is employed to create and return an appropriate image. The lower section demonstrates the image-to-text retrieval process. Here, a user-provided image is matched with images in the database to find the highest similarity. If a high similarity is identified, the pre-stored caption of the matching image is returned. Otherwise, an image captioning model generates and returns a new caption.