ModernBERT + ColBERT: Enhancing biomedical RAG through an advanced re-ranking retriever

TL;DR

The paper tackles biomedical RAG by introducing a two-stage retrieval system that pairs a fast ModernBERT bi-encoder with a precise ColBERTv2 re-ranker, enabling efficient and accurate candidate selection for medical QA. End-to-end fine-tuning with hard negative mining and alignment between retriever and re-ranker yields state-of-the-art MIRAGE average accuracy of 0.4448, outperforming strong baselines while maintaining practical indexing speeds (≈0.80 ms per passage) and competitive inference latency. The approach demonstrates that balancing retrieval speed and token-level semantic matching is crucial in clinical domains, and the results highlight the importance of training strategies and data alignment for two-stage RAG. The work offers a scalable, trustworthy path for deploying biomedical RAG systems in dynamic knowledge environments, though it acknowledges limitations tied to the first-stage recall and outlines concrete avenues for future improvements.

Abstract

Retrieval-Augmented Generation (RAG) is a powerful technique for enriching Large Language Models (LLMs) with external knowledge, allowing for factually grounded responses, a critical requirement in high-stakes domains such as healthcare. However, the efficacy of RAG systems is fundamentally restricted by the performance of their retrieval module, since irrelevant or semantically misaligned documents directly compromise the accuracy of the final generated response. General-purpose dense retrievers can struggle with the nuanced language of specialised domains, while the high accuracy of in-domain models is often achieved at prohibitive computational costs. In this work, we aim to address this trade-off by developing and evaluating a two-stage retrieval architecture that combines a lightweight ModernBERT bidirectional encoder for efficient initial candidate retrieval with a ColBERTv2 late-interaction model for fine-grained re-ranking. We conduct comprehensive evaluations of our retriever module performance and RAG system performance in the biomedical context, fine-tuning the IR module using 10k question-passage pairs from PubMedQA. Our analysis of the retriever module confirmed the positive impact of the ColBERT re-ranker, which improved Recall@3 by up to 4.2 percentage points compared to its retrieve-only counterpart. When integrated into the biomedical RAG, our IR module leads to a state-of-the-art average accuracy of 0.4448 on the five tasks of the MIRAGE question-answering benchmark, outperforming strong baselines such as MedCPT (0.4436). Our ablation studies reveal that this performance is critically dependent on a joint fine-tuning process that aligns the retriever and re-ranker; otherwise, the re-ranker might degrade the performance.

Figures (2)

• Figure 1: Diagram of the two-stage architecture with ModernBERT as the initial retriever and ColBERT as re-ranker.
• Figure 2: Recall@k obtained by base and fine-tuned models in their corresponding configurations with re-ranker (M+C) and without re-ranker (M).