From Evidence-Based Medicine to Knowledge Graph: Retrieval-Augmented Generation for Sports Rehabilitation and a Domain Benchmark

TL;DR

This work tackles the core limitations of medical RAG by integrating evidence-based medicine principles into retrieval, specifically through a PICO-extended knowledge graph and a Bayesian-inspired BETR reranking that accounts for evidence hierarchy without fixed weights. The SR-RAG framework combines PICO-guided HyDE, dual-track evidence recall, and a two-stage reranking pipeline to improve both retrieval quality and factual alignment in sports rehabilitation QA, validated on a large, publicly released knowledge graph and a 1,637-question benchmark. Automated metrics and expert clinician evaluations show high performance across nugget coverage, faithfulness, semantic similarity, and PICOT alignment, while ablation studies underline the importance of the PICO schema and HyDE components. The approach demonstrates portability to other clinical domains and provides a reusable benchmark to address RAG dataset scarcity in sports rehabilitation, advancing practical, evidence-grounded LLM-assisted medical QA.

Abstract

In medicine, large language models (LLMs) increasingly rely on retrieval-augmented generation (RAG) to ground outputs in up-to-date external evidence. However, current RAG approaches focus primarily on performance improvements while overlooking evidence-based medicine (EBM) principles. This study addresses two key gaps: (1) the lack of PICO alignment between queries and retrieved evidence, and (2) the absence of evidence hierarchy considerations during reranking. We present a generalizable strategy for adapting EBM to graph-based RAG, integrating the PICO framework into knowledge graph construction and retrieval, and proposing a Bayesian-inspired reranking algorithm to calibrate ranking scores by evidence grade without introducing predefined weights. We validated this framework in sports rehabilitation, a literature-rich domain currently lacking RAG systems and benchmarks. We released a knowledge graph (357,844 nodes and 371,226 edges) and a reusable benchmark of 1,637 QA pairs. The system achieved 0.830 nugget coverage, 0.819 answer faithfulness, 0.882 semantic similarity, and 0.788 PICOT match accuracy. In a 5-point Likert evaluation, five expert clinicians rated the system 4.66-4.84 across factual accuracy, faithfulness, relevance, safety, and PICO alignment. These findings demonstrate that the proposed EBM adaptation strategy improves retrieval and answer quality and is transferable to other clinical domains. The released resources also help address the scarcity of RAG datasets in sports rehabilitation.

Figures (5)

• Figure 1: Evolution from traditional RAG to graph-based RAG and the proposed EBM-adapted GraphRAG framework integrating evidence hierarchy and the PICO framework.
• Figure 2: Training and validation objective curves for the ordered-grade pairwise calibrator.
• Figure 3: Case-study workflow of SR-RAG, illustrating PICO-guided HyDE, graph retrieval, dual-track evidence recall, reranking with BETR, and structured output.
• Figure 4: Expert clinician evaluation results, including rating distributions and inter-rater disagreement across dimensions.
• Figure 5: Automated and human evaluation pipelines of SR-RAG.