Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale

TL;DR

This work reveals that publicly available text-to-image systems encode and propagate broad social stereotypes across identities, occupations, and everyday objects. Using Stable Diffusion as a primary case, it combines qualitative exemplars with CLIP-based analyses to show stereotype amplification even with neutral prompts, and demonstrates persistent biases despite mitigation efforts and guardrails as seen in DALL·E. The findings highlight the mass-distribution risks of stereotype-laden imagery, tying technical biases to social harms and emphasizing the need for ongoing, multidisciplinary mitigation and governance beyond prompt engineering. The study calls for careful consideration of deployment contexts and long-term strategies to curb the societal harms of language-vision models.

Abstract

Machine learning models that convert user-written text descriptions into images are now widely available online and used by millions of users to generate millions of images a day. We investigate the potential for these models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects. For example, we find cases of prompting for basic traits or social roles resulting in images reinforcing whiteness as ideal, prompting for occupations resulting in amplification of racial and gender disparities, and prompting for objects resulting in reification of American norms. Stereotypes are present regardless of whether prompts explicitly mention identity and demographic language or avoid such language. Moreover, stereotypes persist despite mitigation strategies; neither user attempts to counter stereotypes by requesting images with specific counter-stereotypes nor institutional attempts to add system ``guardrails'' have prevented the perpetuation of stereotypes. Our analysis justifies concerns regarding the impacts of today's models, presenting striking exemplars, and connecting these findings with deep insights into harms drawn from social scientific and humanist disciplines. This work contributes to the effort to shed light on the uniquely complex biases in language-vision models and demonstrates the ways that the mass deployment of text-to-image generation models results in mass dissemination of stereotypes and resulting harms.

Figures (14)

• Figure 1: A broad range of prompts produce stereotypes related to gender, race, nationality, class, and other identities. Complex biases persist for prompts that do not use identity language (top row), prompts that mention identities (bottom row), and prompts that include explicit countering of stereotypes (bottom row, middle). We present two random examples for each prompt.
• Figure 2: Simple user prompts generate thousands of images perpetuating dangerous stereotypes. For each descriptor, the prompt "A photo of the face of [DESCRIPTOR]" is fed to Stable Diffusion, and we present a random sample of 10 images generated by the Stable Diffusion model. We find that the produced images define attractiveness as near the "White ideal" nla.cat-vn136537 and tie emotionality specifically to stereotypically white feminine features. Meanwhile, the images exoticize people with darker skin tone, non-European adornment, and Afro-ethnic hair doi:10.1080/01419870601143992. A thug generates faces with dark skin tone and stereotypically masculine, African-American features doi:10.1177/107769909607300410, and a terrorist generates brown faces with dark hair and beards, consistent with the American narrative that terrorists are brown men with beards Corbin2017TerroristsAA. Images of social structures, like a happy family, perpetuate a singular, heteronormative notion of family. All images are randomly sampled from 100 generated outputs.
• Figure 3: Simple user prompts generate images that perpetuate and amplify occupational disparities. Images generated using the prompt "A photo of the face of [OCCUPATION]" amplify gender and race imbalances across occupations. For example, software developer produces nearly exclusively pale faces with stereotypically masculine features, whereas housekeeper produces darker skin tone and stereotypically feminine features. All images are randomly sampled from 100 generated outputs.
• Figure 4: Quantifying stereotype amplification. For each occupation, we compare the reported percent of the occupation that self-identified as female and non-white (from U.S. Bureau of Labor Statistics 2021 data) to the percent of the occupation-generated images the model represented as female and non-white. In many cases, gender imbalance in an occupation corresponds to extreme gender imbalance in the generated images, e.g. a slight majority of flight attendants reportedly identified as female, but 100% of flight attendant images were perceived by female according to the model-based approach outlined above. Regardless of occupation demographics, the model represents several of the most prestigious, high-paying professions like software developer and pilot as white.
• Figure 5: Generated images of everyday objects encode persistent stereotypes. The images generated from prompts with no identity descriptor perpetuate North American norms of objects' appearances: these neutral prompts are typically extremely similar to images generated from prompts with "North America" (top row). These are most different from prompts with "Africa" (bottom row), which encode harmful stereotypes of poverty. We present two random examples for each prompt.