New Job, New Gender? Measuring the Social Bias in Image Generation Models

TL;DR

BiasPainter addresses the lack of accurate, scalable bias evaluation in image-generation systems by automating bias triggering through seed images and neutral prompts. It systematically measures race, gender, and age changes via automated physical-attribute analyses and computes image-, word-, and model-level bias scores, validated across six major generation systems with a 90.8% human-annotated accuracy. The framework enables both diagnostic bias discovery and mitigation evaluation, showing that prompt design can reduce bias though not eliminate it entirely. The work contributes a reusable evaluation pipeline, a sizable seed/prompts dataset, and supporting materials to advance fair development in image-generation technologies. Its practical impact lies in enabling developers and researchers to quantify biases, compare models, and iteratively design mitigation strategies with measurable fairness improvements.

Abstract

Image generation models can generate or edit images from a given text. Recent advancements in image generation technology, exemplified by DALL-E and Midjourney, have been groundbreaking. These advanced models, despite their impressive capabilities, are often trained on massive Internet datasets, making them susceptible to generating content that perpetuates social stereotypes and biases, which can lead to severe consequences. Prior research on assessing bias within image generation models suffers from several shortcomings, including limited accuracy, reliance on extensive human labor, and lack of comprehensive analysis. In this paper, we propose BiasPainter, a novel evaluation framework that can accurately, automatically and comprehensively trigger social bias in image generation models. BiasPainter uses a diverse range of seed images of individuals and prompts the image generation models to edit these images using gender, race, and age-neutral queries. These queries span 62 professions, 39 activities, 57 types of objects, and 70 personality traits. The framework then compares the edited images to the original seed images, focusing on the significant changes related to gender, race, and age. BiasPainter adopts a key insight that these characteristics should not be modified when subjected to neutral prompts. Built upon this design, BiasPainter can trigger the social bias and evaluate the fairness of image generation models. We use BiasPainter to evaluate six widely-used image generation models, such as stable diffusion and Midjourney. Experimental results show that BiasPainter can successfully trigger social bias in image generation models. According to our human evaluation, BiasPainter can achieve 90.8% accuracy on automatic bias detection, which is significantly higher than the results reported in previous work.

Figures (5)

• Figure 1: Examples of Biased Generation Detected by BiasPainter.
• Figure 2: The Overview Framework of BiasPainter
• Figure 3: Image Processing Pipeline to Access the Skin Tone Information
• Figure 4: Visualization of Profession Word Bias Scores in Stable Diffusion 1.5
• Figure 5: The Seed Image Examples