Stable Diffusion Exposed: Gender Bias from Prompt to Image
Yankun Wu, Yuta Nakashima, Noa Garcia
TL;DR
This work tackles gender bias in text-to-image generation by introducing an automated, multi-stage evaluation protocol for Stable Diffusion. It leverages free-form triplet prompts (neutral, feminine, masculine) to trace bias from prompt space through denoising and image spaces, revealing that neutral prompts tend to resemble masculine prompts and that objects not explicitly mentioned in prompts exhibit gendered generation patterns. The authors identify the bias source in text embeddings and propagate it via the generation pipeline, analyze object-level and dependency-based disparities using advanced grounding and attention techniques, and provide actionable recommendations for developers and users to mitigate bias. The protocol enables systematic, automatic bias analysis across prompts, representations, and generated images, contributing a practical framework for reducing gender disparities in diffusion-based image synthesis. Overall, the study demonstrates robust, cross-model, cross-dataset evidence of gender-influenced object generation and layouts, and offers concrete debiasing strategies and user guidance to improve fairness in practice.
Abstract
Several studies have raised awareness about social biases in image generative models, demonstrating their predisposition towards stereotypes and imbalances. This paper contributes to this growing body of research by introducing an evaluation protocol that analyzes the impact of gender indicators at every step of the generation process on Stable Diffusion images. Leveraging insights from prior work, we explore how gender indicators not only affect gender presentation but also the representation of objects and layouts within the generated images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts. We also reveal that neutral prompts tend to produce images more aligned with masculine prompts than their feminine counterparts. We further explore where bias originates through representational disparities and how it manifests in the images via prompt-image dependencies, and provide recommendations for developers and users to mitigate potential bias in image generation.