BiasConnect: Investigating Bias Interactions in Text-to-Image Models

TL;DR

BiasConnect introduces a counterfactual, causal-analysis framework to quantify how biases across axes in Text-to-Image models interact, using counterfactual prompts, VQA-based attribute extraction, and pairwise causal discovery to build directed bias graphs. It then quantifies mitigation effects with the Intersectional Sensitivity score, validated by a strong correlation ($W_1$-based IS) of $+0.696$ with post-mitigation outcomes and demonstrated robustness across image counts and VQA noise. The approach enables prompt- and model-level audits, comparison against real-world distributions, and identification of mitigation strategies that account for intersectionality. This work advances fairer generative modeling by revealing and leveraging interdependencies among biases rather than treating them in isolation, with practical implications for model selection and bias-aligned data curation.

Abstract

The biases exhibited by Text-to-Image (TTI) models are often treated as if they are independent, but in reality, they may be deeply interrelated. Addressing bias along one dimension, such as ethnicity or age, can inadvertently influence another dimension, like gender, either mitigating or exacerbating existing disparities. Understanding these interdependencies is crucial for designing fairer generative models, yet measuring such effects quantitatively remains a challenge. In this paper, we aim to address these questions by introducing BiasConnect, a novel tool designed to analyze and quantify bias interactions in TTI models. Our approach leverages a counterfactual-based framework to generate pairwise causal graphs that reveals the underlying structure of bias interactions for the given text prompt. Additionally, our method provides empirical estimates that indicate how other bias dimensions shift toward or away from an ideal distribution when a given bias is modified. Our estimates have a strong correlation (+0.69) with the interdependency observations post bias mitigation. We demonstrate the utility of BiasConnect for selecting optimal bias mitigation axes, comparing different TTI models on the dependencies they learn, and understanding the amplification of intersectional societal biases in TTI models.

Figures (8)

• Figure 1: An example output of BiasConnect , revealing the negative impact of bias mitigation along one dimension on another dimension. Here, increasing the gender diversity (GEN) skews age distribution (AGE) for images of musicians generated by Stable Diffusion 1.4 rombach2022high.
• Figure 2: An overview of BiasConnect . We use a counterfactual-based approach to measure pairwise causality between bias axes. For dependent axes, we measure the causal effect, estimating how bias mitigation on one axis impacts another.
• Figure 3: The figure illustrates bias interpretations from Bias Connects, combining all pairwise graphs into one. (a) Shows how mitigating clothing bias also mitigates emotion bias. (b) Explores interactions between non-traditional bias axes in the TIBET dataset. (c) Reveals that generating ethnically diverse athletes reduces gender diversity. (d) Demonstrates that diversifying salesperson clothing is best achieved by increasing ethnic diversity rather than directly specifying clothing variation.
• Figure 4: Sensitivity analysis on BiasConnect . . We evaluate the robustness of our approach by analyzing the impact of VQA errors and the effect of the number of images on the pairwise causal graph and Intersectional Sensitivity .
• Figure 5: We compare aggregated causal graphs for four models: Stable Diffusion 1.4, Flux-dev, Kandinsky 2.2, and Playground 2.5. These graphs combine pairwise causal relationships across all bias axes, accumulated from occupation prompts in our dataset.