Bias Evaluation and Mitigation in Retrieval-Augmented Medical Question-Answering Systems

TL;DR

This study systematically evaluates demographic biases within medical RAG pipelines across multiple QA benchmarks, including MedQA, MedMCQA, MMLU, and EquityMedQA, and quantifies disparities in retrieval consistency and answer correctness by generating and analyzing queries sensitive to demographic variations.

Abstract

Medical Question Answering systems based on Retrieval Augmented Generation is promising for clinical decision support because they can integrate external knowledge, thus reducing inaccuracies inherent in standalone large language models (LLMs). However, these systems may unintentionally propagate or amplify biases associated with sensitive demographic attributes like race, gender, and socioeconomic factors. This study systematically evaluates demographic biases within medical RAG pipelines across multiple QA benchmarks, including MedQA, MedMCQA, MMLU, and EquityMedQA. We quantify disparities in retrieval consistency and answer correctness by generating and analyzing queries sensitive to demographic variations. We further implement and compare several bias mitigation strategies to address identified biases, including Chain of Thought reasoning, Counterfactual filtering, Adversarial prompt refinement, and Majority Vote aggregation. Experimental results reveal significant demographic disparities, highlighting that Majority Vote aggregation notably improves accuracy and fairness metrics. Our findings underscore the critical need for explicitly fairness-aware retrieval methods and prompt engineering strategies to develop truly equitable medical QA systems.

Figures (1)

• Figure 1: Overview of our proposed bias mitigation framework for medical Retrieval-Augmented Generation (RAG), highlighting three effective filtering methods and Majority Vote aggregation. The system takes an input medical query, retrieves relevant documents from a knowledge base (e.g., PubMed, textbooks, Wikipedia) using a retriever (Contriever BM25), and applies multiple filtering strategies before generating an answer. Three bias mitigation techniques are incorporated: (1) Chain of Thought (COT) Filtering, which encourages structured, evidence-based reasoning while avoiding implicit biases; (2) Counterfactual Filtering, which generates responses from different demographic perspectives and ensures consistency in scientific accuracy; and (3) Adversarial Prompt Filtering, which identifies and avoids biased phrasing in model-generated responses. Finally, a Majority Vote mechanism aggregates multiple responses to mitigate potential biases further and improve answer robustness.