Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domain

TL;DR

This paper addresses how to build effective, efficient retrieval-augmented generation (RAG) systems for medical-domain tasks. It analyzes each RAG component (query classification, chunking, indexing, retrieval, and response prompting), proposing practical alternatives and a three-step evaluation strategy to identify optimal configurations. Through extensive experiments across open-domain and biomedical benchmarks, the authors identify BP-RAG—characterized by small2big chunking, hybrid indexing, BGE-base embeddings, query classification, pseudo-response augmentation, and COT-Refine prompting—as a strong, practical setup for industrial deployments. The findings offer concrete guidance for practitioners balancing performance and latency, while acknowledging limitations (use of open-source LLMs) and ethical considerations for real-world medical applications.

Abstract

While retrieval augmented generation (RAG) has been swiftly adopted in industrial applications based on large language models (LLMs), there is no consensus on what are the best practices for building a RAG system in terms of what are the components, how to organize these components and how to implement each component for the industrial applications, especially in the medical domain. In this work, we first carefully analyze each component of the RAG system and propose practical alternatives for each component. Then, we conduct systematic evaluations on three types of tasks, revealing the best practices for improving the RAG system and how LLM-based RAG systems make trade-offs between performance and efficiency.

Figures (1)

• Figure 1: The workflow of retrieval-augmented generation considered in this study. We investigate the contribution of each module and provide insights into optimal RAG practices through extensive experiments. The methods selected for the best practice are underlined.