OpenBias: Open-set Bias Detection in Text-to-Image Generative Models

TL;DR

OpenBias tackles open-set bias detection in text-to-image generation by using an LLM to propose biases from captions, generating images with the same prompts, and evaluating bias presence with a VQA model. It introduces an entropy-based bias severity score and distinguishes context-aware and context-free bias, enabling robust cross-bias ranking. The authors validate OpenBias on Stable Diffusion variants, showing alignment with closed-set baselines and human judgments, and demonstrate discovery of novel biases beyond predefined sets. The approach is modular and model-agnostic, suitable for integration with existing bias mitigation pipelines and extensions to captionless or unconditional generators.

Abstract

Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply investigate their safety and fairness to not disseminate and perpetuate any kind of biases. However, existing works focus on detecting closed sets of biases defined a priori, limiting the studies to well-known concepts. In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set. OpenBias has three stages. In the first phase, we leverage a Large Language Model (LLM) to propose biases given a set of captions. Secondly, the target generative model produces images using the same set of captions. Lastly, a Vision Question Answering model recognizes the presence and extent of the previously proposed biases. We study the behavior of Stable Diffusion 1.5, 2, and XL emphasizing new biases, never investigated before. Via quantitative experiments, we demonstrate that OpenBias agrees with current closed-set bias detection methods and human judgement.

Figures (26)

• Figure 1: OpenBias discovers biases in T2I models within an open-set scenario. In contrast to previous works fairDiffusion2023ITIGEN_2023_ICCVkenfack2022repfairgan, our pipeline does not require a predefined list of biases but proposes a set of novel domain-specific biases.
• Figure 2: OpenBias pipeline. Starting with a dataset of real textual captions ($\mathcal{T}$) we leverage a Large Language Model (LLM) to build a knowledge base $\mathcal{B}$ of possible biases that may occur during the image generation process. In the second stage, synthesized images are generated using the target generative model conditioned on captions where a potential bias has been identified. Finally, the biases are assessed and quantified by querying a VQA model with caption-specific questions extracted during the bias proposal phase.
• Figure 3: Comparison of context-aware discovered biases on Stable Diffusion XL, 2 and 1.5 podell2023sdxlLDM_2022_CVPR with captions from COCO DBLP:journals/corr/LinMBHPRDZ14.
• Figure 4: Comparison of context-aware found biases on Stable Diffusion XL, 2 and 1.5 podell2023sdxlLDM_2022_CVPR on captions from Flick30k young-etal-2014-image.
• Figure 5: Human evaluation results.