OASIS Uncovers: High-Quality T2I Models, Same Old Stereotypes

TL;DR

This work defines a sociologically aligned measure of stereotypes in text-to-image models and introduces OASIS, a toolbox with four components: Stereotype Score (M1), Weighted Alignment Score (WAlS, M2), Stereotypes from Optimized Prompts (StOP, U1), and Stereotype Propagation Index (SPI, U2). It leverages open-set stereotype candidates generated by an LLM and CLIP-based scoring to quantify distributional and spectral biases and to trace their origins in model internals and generation-time dynamics. Applying OASIS to SDv2, SDv3, and FLUX.1 reveals that newer models still harbor strong stereotypical predispositions, with higher biases for under-represented nationalities, and demonstrates trade-offs between reducing stereotypes and maintaining attribute variance. The findings underscore the need for model auditing, dedicated mitigation strategies, and inclusive data practices in the development of robust, fair T2I systems.

Abstract

Images generated by text-to-image (T2I) models often exhibit visual biases and stereotypes of concepts such as culture and profession. Existing quantitative measures of stereotypes are based on statistical parity that does not align with the sociological definition of stereotypes and, therefore, incorrectly categorizes biases as stereotypes. Instead of oversimplifying stereotypes as biases, we propose a quantitative measure of stereotypes that aligns with its sociological definition. We then propose OASIS to measure the stereotypes in a generated dataset and understand their origins within the T2I model. OASIS includes two scores to measure stereotypes from a generated image dataset: (M1) Stereotype Score to measure the distributional violation of stereotypical attributes, and (M2) WALS to measure spectral variance in the images along a stereotypical attribute. OASIS also includes two methods to understand the origins of stereotypes in T2I models: (U1) StOP to discover attributes that the T2I model internally associates with a given concept, and (U2) SPI to quantify the emergence of stereotypical attributes in the latent space of the T2I model during image generation. Despite the considerable progress in image fidelity, using OASIS, we conclude that newer T2I models such as FLUX.1 and SDv3 contain strong stereotypical predispositions about concepts and still generate images with widespread stereotypical attributes. Additionally, the quantity of stereotypes worsens for nationalities with lower Internet footprints.

Figures (11)

• Figure 1: Measuring Stereotypes in Text-to-Image Models. (a) The images generated by T2I models corresponding to the prompt "A photo of an Iranian person" overwhelmingly contain stereotypical tropes such as beard, turban, and religious attire although the prompt is devoid of this information. (b) The proposed toolbox OASIS includes complementary methods for quantifying stereotypes. Stereotype Score measures the over-representation of stereotypical attributes while WAlS measures the variance of images along these attributes. (c) SPI quantifies the emergence of stereotypes from the latent space of these models and helps understand the origin of stereotypes within a T2I model.
• Figure 2: An overview of OASIS. Given a text prompt, a set of images is generated using the T2I model $\mathcal{M}$. Simultaneously, a stereotype candidate set is created using an LLM. OASIS then performs four quantitative analyses: (M1) Stereotype Score $\Psi$ to measure stereotypes based on \ref{['def:stereo']}, (M2) WAlS to assess the spectral variance of $\mathcal{D}$ w.r.t. a stereotypical attribute, (U1) StOP to discover the stereotypical attributes that $\mathcal{M}$ associates with a given concept, and (U2) SPI to quantify the emergence of stereotypical attributes in the latent space of $\mathcal{M}$ during image generation.
• Figure 3: Comparing stereotype scores for nationalities against the number of Internet users shows that stereotypes are higher for under-represented nationalities.
• Figure 4: WAlS: Comparison of SDv2, SDv3, and $\text{FLUX.1}$ on spectral variance in the generated images across different attributes, calculated for Iranian, Mexican, and Indian nationalities.
• Figure 5: Comparison of T2I models based on stereotype scores and WAlS on three nationalities. Different colors show different T2I models and the shapes of the markers denote the attributes.