Mitigating Bias in RAG: Controlling the Embedder

TL;DR

This work tackles bias in retrieval augmented generation (RAG) by modeling how biases from the embedder, LLM, and corpus interact. It introduces a bias metric for each component and a linear model, $R_b = s\cdot E_b + L_b + \epsilon$, where $s$ is the sensitivity of the RAG to embedder bias, $L_b$ is the LLM bias, and $\epsilon$ captures extraneous knowledge conflict. Through extensive experiments across 6 LLMs and 120 embedders using gender and political bias case studies (GenderBias-QA and PoliticBias-QA), the authors demonstrate that reverse-biasing the embedder can debias the overall RAG system with minimal loss in utility, and that sensitivity varies by bias type and model. They further show that a linear relationship between embedder bias and RAG bias persists under fixed corpora and persists across several test conditions, enabling robust debiasing strategies such as fine-tuning and projection-based methods, though stochastic sampling offers limited relief. The results underscore the practical insight that addressing bias via the embedder — rather than solely the LLM or corpus — can yield fairer RAG systems, while also outlining limitations and directions for extending the analysis to more complex, real-world scenarios.

Abstract

In retrieval augmented generation (RAG) systems, each individual component -- the LLM, embedder, and corpus -- could introduce biases in the form of skews towards outputting certain perspectives or identities. In this work, we study the conflict between biases of each component and their relationship to the overall bias of the RAG system, which we call bias conflict. Examining both gender and political biases as case studies, we show that bias conflict can be characterized through a linear relationship among components despite its complexity in 6 different LLMs. Through comprehensive fine-tuning experiments creating 120 differently biased embedders, we demonstrate how to control bias while maintaining utility and reveal the importance of reverse-biasing the embedder to mitigate bias in the overall system. Additionally, we find that LLMs and tasks exhibit varying sensitivities to the embedder bias, a crucial factor to consider for debiasing. Our results underscore that a fair RAG system can be better achieved by carefully controlling the bias of the embedder rather than increasing its fairness.

Figures (12)

• Figure 1: RAG System. A RAG system consists of the LLM, embedder, and corpus. Given a query as input, the embedder retrieves documents from the corpus that are similar to the query. The LLM takes as input the query and retrieved document to generate an output. Each component introduces bias into the system which propagates into latter stages. We find that the change in RAG bias ($s \cdot \Delta b$ ) scales linearly with the change in embedder bias ($\Delta b$ ), as shown in Figure \ref{['fig:training']}.
• Figure 2: Pareto Frontier of Fine-tuning. Pareto frontier showing the trade-off between bias and accuracy for validation. The bias of the fine-tuned embedders first start increasing towards females and conservative views before losing performance on RAG Mini-Wikipedia. With light fine-tuning, it is possible to reverse bias the embedder with minimal loss in utility.
• Figure 3: Controlling Bias through Fine-tuning. Linear relationship between the RAG bias ($R_b$) and embedder bias ($E_b$) for the 20 embedders. If the sensitivity $s$ is sufficiently high, it is possible to debias the entire RAG system ($R_b=0$). Results for all 6 LLMs are in §\ref{['app:six-llms']}.
• Figure 4: Corpus Bias. RAG bias ($R_b$) when the corpus bias ($C_b$) changes for three different embedders. The base embedder is GTE-base, the optimal embedder is the embedder that results in $R_b \approx 0$ with a neutral corpus ($C_b$), and the degenerate embedder is a heavily reverse biased embedder. The RAG bias scales linearly with the corpus bias for the base and optimal embedder while the linearity breaks as the embedder becomes more degenerate.
• Figure 5: Controlling bias through Fine-tuning. There is a linear relationship between the RAG bias and embedder bias. It is possible to debias the entire RAG system if the sensitivity $s$ is sufficiently high.