PICOs-RAG: PICO-supported Query Rewriting for Retrieval-Augmented Generation in Evidence-Based Medicine

TL;DR

This work targets the problem of informal, incomplete medical queries that hamper retrieval quality in evidence-based medicine. It introduces PICOs-RAG, a pipeline that classifies queries by medical discipline, expands and normalizes them, and extracts PICO elements to drive precise evidence retrieval, which then informs the final answer generation. The approach yields improvements in both accuracy and relevance over baselines, validated through dual evaluation methods and ablation studies. The method has practical significance for safer, more reliable AI-supported medical decision-making in real-world clinical settings.

Abstract

Evidence-based medicine (EBM) research has always been of paramount importance. It is important to find appropriate medical theoretical support for the needs from physicians or patients to reduce the occurrence of medical accidents. This process is often carried out by human querying relevant literature databases, which lacks objectivity and efficiency. Therefore, researchers utilize retrieval-augmented generation (RAG) to search for evidence and generate responses automatically. However, current RAG methods struggle to handle complex queries in real-world clinical scenarios. For example, when queries lack certain information or use imprecise language, the model may retrieve irrelevant evidence and generate unhelpful answers. To address this issue, we present the PICOs-RAG to expand the user queries into a better format. Our method can expand and normalize the queries into professional ones and use the PICO format, a search strategy tool present in EBM, to extract the most important information used for retrieval. This approach significantly enhances retrieval efficiency and relevance, resulting in up to an 8.8\% improvement compared to the baseline evaluated by our method. Thereby the PICOs-RAG improves the performance of the large language models into a helpful and reliable medical assistant in EBM.

Figures (8)

• Figure 1: The architecture of current RAG methods compared with our PICOs method. The PICOs enhance the relevance of the evidence retrieval process and improve the accuracy of responses, enabling the LLM to provide precise and targeted assistance.
• Figure 2: Steps of EBM. EBM primarily involves formulating the queries, searching for all possible relevant evidence, filtering and reordering the obtained evidence, and answering the question based on the evidence gathered. Our method focuses on optimizing the query, addressing the issue where queries may lack professionalism and clarity due to insufficient knowledge methley2014pico.
• Figure 3: PICOs method to help RAG used on EBM. PICOs first capture the objective aspects of the queries. The classification results then prompt the LLM to expand and refine the query. After the query is completed, we use the LLM to extract the PICO elements from the question. These PICO elements are then passed to the retrieval module for more precise extraction. The retrieved results, along with the refined question, are then fed into the response model to generate the final response.
• Figure 4: Examples of the method we expand and normalize query. The sentences rewritten by the LLM use more professional language, and the user's query is effectively completed.
• Figure 5: Examples of the method we extract the PICOs from the rewritten query. The PICO-formatted text extracted through PICOs is more targeted compared to the expanded sentences, resulting in content retrieval that is more closely aligned with the query.