Interactive Discovery and Exploration of Visual Bias in Generative Text-to-Image Models

TL;DR

Bias in generative Text-to-Image models is pervasive and difficult to analyze systematically. ViBEx presents an interactive, model-agnostic workflow that combines a prompting-tree interface, zero-shot bias probing via CLIP, and confirmatory visual analytics to discover and validate visual biases in T2I outputs. The approach is demonstrated through expert case studies that reveal novel biases in Stable Diffusion 3 and shows how Bayes-based bias quantification and CLIP similarities can guide rapid, human-in-the-loop auditing. Overall, ViBEx provides a practical tool for bias discovery, documentation, and auditing in generative vision systems, with potential pathways toward mitigation and responsible deployment.

Abstract

Bias in generative Text-to-Image (T2I) models is a known issue, yet systematically analyzing such models' outputs to uncover it remains challenging. We introduce the Visual Bias Explorer (ViBEx) to interactively explore the output space of T2I models to support the discovery of visual bias. ViBEx introduces a novel flexible prompting tree interface in combination with zero-shot bias probing using CLIP for quick and approximate bias exploration. It additionally supports in-depth confirmatory bias analysis through visual inspection of forward, intersectional, and inverse bias queries. ViBEx is model-agnostic and publicly available. In four case study interviews, experts in AI and ethics were able to discover visual biases that have so far not been described in literature.

Figures (22)

• Figure 1: The ViBEx workflow: Users define anchor and test concepts and subsequently perform bias queries, resulting in bias candidates. Elements in blue represent a non-real-time operation.
• Figure 2: Schematic overview of the prompting tree. The root node "picture" is part of all prompts. Anchor concepts (in red) represent the classes $c_1, c_2 \in C$, for which we probe different test concepts $t_i \in T$ (blue) for potential bias. The relation type between two concepts is indicated by the edge label. From this tree we may parse prompts such as "picture that shows a young person" or "picture that shows a female person wearing a suit".
• Figure 3: Two prompting trees with the same concepts but differing data sources. The data loaded from the FairFace dataset is gender-balanced. Thus, gender is expected to be neutral. The minimal imbalance hints at a bias in CLIP. The SD3 data shows a pronounced bias toward "caucasian" for the test concept "male".
• Figure 4: Strip plots for three test concepts ("gray hair", "painted picture", "black person"). The univariate distribution of image text similarities is plotted for each anchor concept $c_i \in C$, here "woman" and "man". Note how the highest scoring image for "black person" is a silhouette picture.
• Figure 5: Intersectional bias plot for "woman" and "traditional clothing" with the anchor concepts "Germany" and "Nigeria". Images in the top right are Nigerian women in traditional clothing, while for Germany no traditional clothing is present.