Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systems

TL;DR

This work investigates fairness in Retrieval-Augmented Generation (RAG) for open-domain QA by introducing a scenario-based QA fairness framework and evaluating across multiple RAG architectures using the TREC 2022 Fair Ranking Track and BBQ datasets. It analyzes fairness contributions from each component of the RAG pipeline (Retriever, Refiner, Judger, Generator) and demonstrates that improvements in utility metrics such as $EM$ or $ROUGE$-1 do not reliably translate to fairer outputs, highlighting a persistent bias across retrieval and generation stages. The study employs the FlashRAG toolkit and reports results on a public benchmark, revealing that dense retrieval with carefully balanced document ranking can mitigate some disparities, yet larger generator models can amplify bias. The findings underscore the need for holistic, component-aware fairness interventions in RAG systems and lay groundwork for fairness-aware retrieval, ranking, and generation techniques with practical implications for real-world QA systems.

Abstract

Retrieval-Augmented Generation (RAG) has recently gained significant attention for its enhanced ability to integrate external knowledge sources into open-domain question answering (QA) tasks. However, it remains unclear how these models address fairness concerns, particularly with respect to sensitive attributes such as gender, geographic location, and other demographic factors. First, as language models evolve to prioritize utility, like improving exact match accuracy, fairness considerations may have been largely overlooked. Second, the complex, multi-component architecture of RAG methods poses challenges in identifying and mitigating biases, as each component is optimized for distinct objectives. In this paper, we aim to empirically evaluate fairness in several RAG methods. We propose a fairness evaluation framework tailored to RAG, using scenario-based questions and analyzing disparities across demographic attributes. Our experimental results indicate that, despite recent advances in utility-driven optimization, fairness issues persist in both the retrieval and generation stages. These findings underscore the need for targeted interventions to address fairness concerns throughout the RAG pipeline. The dataset and code used in this study are publicly available at this GitHub Repository https://github.com/elviswxy/RAG_fairness .

Figures (10)

• Figure 1: Illustration of two scenarios of RAG: (a) RAG enhances both the accuracy and fairness and (b) RAG maintains answer accuracy but not fairness. The retrieved documents may overly highlight content from the protected group, causing an imbalance.
• Figure 2: Proposed RAG fairness evaluation framework, showing the flow from data construction collection to performance evaluation.
• Figure 3: Evaluation of EM and $\text{GD}_\text{S1}$ for retrievers, with a focus on different retrieval methods (BM25, E5-base, and E5-large) and varying retrieval document numbers (ret_num = 1, 2, 5).
• Figure 4: Evaluation of EM and $\text{GD}_\text{S1}$ for Selective-Context and Iter-RetGen Refiner.
• Figure 5: Evaluation of EM and $\text{GD}_\text{S1}$ for FLARE and SKR judgers. Since Zero-shot and Naive do not use a judger component, their $\text{GD}_\text{S1}$ values are set to zero.