Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models

TL;DR

This work tackles the problem that text-to-image biases across attributes such as gender, ethnicity, and age interact in complex, intersectional ways. It introduces BiasConnect to quantify these interactions via counterfactual prompts and a Wasserstein-based Intersectional Sensitivity measure, and Bias Intersectionality Matrix visualizations. It then presents InterMit, a modular, user-guided, training-free framework that uses the intersectionality map to perform efficient joint bias mitigation with user-defined target distributions and priorities. The authors validate the approach on multiple TTI models and datasets, showing that BiasConnect’s IS correlates with outcome changes and that InterMit can achieve greater bias reduction with fewer mitigation steps and better image quality, while warning about trade-offs among axes. This work provides a practical toolkit for auditing and mitigating intersectional biases in generative vision-language systems.

Abstract

The biases exhibited by text-to-image (TTI) models are often treated as independent, though in reality, they may be deeply interrelated. Addressing bias along one dimension - such as ethnicity or age - can inadvertently affect another, like gender, either mitigating or exacerbating existing disparities. Understanding these interdependencies is crucial for designing fairer generative models, yet measuring such effects quantitatively remains a challenge. To address this, we introduce BiasConnect, a novel tool for analyzing and quantifying bias interactions in TTI models. BiasConnect uses counterfactual interventions along different bias axes to reveal the underlying structure of these interactions and estimates the effect of mitigating one bias axis on another. These estimates show strong correlation (+0.65) with observed post-mitigation outcomes. Building on BiasConnect, we propose InterMit, an intersectional bias mitigation algorithm guided by user-defined target distributions and priority weights. InterMit achieves lower bias (0.33 vs. 0.52) with fewer mitigation steps (2.38 vs. 3.15 average steps), and yields superior image quality compared to traditional techniques. Although our implementation is training-free, InterMit is modular and can be integrated with many existing debiasing approaches for TTI models, making it a flexible and extensible solution.

Figures (7)

• Figure 1: An example for which BiasConnect estimates a negative impact of bias mitigation along one axis on another axis. For this query, increasing the gender diversity (Gen) skews age distribution (Age) for images of musicians generated by Flux-dev.
• Figure 2: An overview of BiasConnect . We use a counterfactual-based approach to measure how interventions along a single bias axes impact other bias axes. Our metric Intersectional Sensitivity estimates how bias mitigation on one axis impacts another. Our results are visualized as a a matrix called the Bias Intersectionality Matrix.
• Figure 3: Analyzing bias intersectionality matrices from BiasConnect . (a) Shows how mitigating clothing bias also mitigates emotion bias. (b) Explores interactions between non-traditional bias axes in the TIBET dataset. (c) Reveals that generating ethnically diverse athletes reduces gender diversity. BiasConnect can allow the user the user to understand whether interventions along one dimension impact other dimensions positively or negatively.
• Figure 4: Sensitivity analysis on BiasConnect . We evaluate the robustness of our approach by analyzing the impact of VQA errors and the effect of the number of images on Intersectional Sensitivity .
• Figure 5: Three examples of mitigation using InterMit . Priority vectors guide the mitigation process. Columns that are a part of sub-matrix $\mathbf{S}'$ are in blue. As shown in (a) and (c), the algorithm mitigates multiple biases in fewer steps. (b) shows how user-defined priorities guide the process and when thresholds are met. Mitigating one axis, like ethnicity (Eth), can also affect others like clothing (Clo) and emotion (Emo), revealing bias interdependencies.