Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach

Sourojit Ghosh, Pranav Narayanan Venkit, Sanjana Gautam, Shomir Wilson, Aylin Caliskan

TL;DR

This study investigates how text-to-image generative models reproduce or distort non-Western cultures, focusing on Indian contexts, by engaging five Indian subculture groups (n=25) in live-tprompted outputs and grounded theory analysis. It identifies two novel representational harms—exoticism and cultural misappropriation—alongside established harms such as stereotyping, erasure, disparagement, and quality-of-service disparities, driven by a Western gaze and North Indian default depictions. The authors propose sociotechnical, community-centered design principles and context-specific guidelines to create culturally sensitive T2Is, including culture-aware data practices, decolonial framing, and model fine-tuning for subcultures. The work advances equitable AI by foregrounding Global South perspectives, offering actionable design considerations that can be adapted to other non-Western cultures and non-Western languages to reduce representational harms in GAI outputs.

Abstract

Our research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures, with a focus on Indian contexts. Despite the transformative potential of T2Is in content creation, concerns have arisen regarding biases that may lead to misrepresentations and marginalizations. Through a community-centered approach and grounded theory analysis of 5 focus groups from diverse Indian subcultures, we explore how T2I outputs to English prompts depict Indian culture and its subcultures, uncovering novel representational harms such as exoticism and cultural misappropriation. These findings highlight the urgent need for inclusive and culturally sensitive T2I systems. We propose design guidelines informed by a sociotechnical perspective, aiming to address these issues and contribute to the development of more equitable and representative GAI technologies globally. Our work also underscores the necessity of adopting a community-centered approach to comprehend the sociotechnical dynamics of these models, complementing existing work in this space while identifying and addressing the potential negative repercussions and harms that may arise when these models are deployed on a global scale.

Do Generative AI Models Output Harm while Representing Non-Western Cultures: Evidence from A Community-Centered Approach

TL;DR

This study investigates how text-to-image generative models reproduce or distort non-Western cultures, focusing on Indian contexts, by engaging five Indian subculture groups (n=25) in live-tprompted outputs and grounded theory analysis. It identifies two novel representational harms—exoticism and cultural misappropriation—alongside established harms such as stereotyping, erasure, disparagement, and quality-of-service disparities, driven by a Western gaze and North Indian default depictions. The authors propose sociotechnical, community-centered design principles and context-specific guidelines to create culturally sensitive T2Is, including culture-aware data practices, decolonial framing, and model fine-tuning for subcultures. The work advances equitable AI by foregrounding Global South perspectives, offering actionable design considerations that can be adapted to other non-Western cultures and non-Western languages to reduce representational harms in GAI outputs.

Abstract

Our research investigates the impact of Generative Artificial Intelligence (GAI) models, specifically text-to-image generators (T2Is), on the representation of non-Western cultures, with a focus on Indian contexts. Despite the transformative potential of T2Is in content creation, concerns have arisen regarding biases that may lead to misrepresentations and marginalizations. Through a community-centered approach and grounded theory analysis of 5 focus groups from diverse Indian subcultures, we explore how T2I outputs to English prompts depict Indian culture and its subcultures, uncovering novel representational harms such as exoticism and cultural misappropriation. These findings highlight the urgent need for inclusive and culturally sensitive T2I systems. We propose design guidelines informed by a sociotechnical perspective, aiming to address these issues and contribute to the development of more equitable and representative GAI technologies globally. Our work also underscores the necessity of adopting a community-centered approach to comprehend the sociotechnical dynamics of these models, complementing existing work in this space while identifying and addressing the potential negative repercussions and harms that may arise when these models are deployed on a global scale.
Paper Structure (25 sections, 9 figures, 1 table)

This paper contains 25 sections, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Biases and Harms within GAI Tools: A Sociotechnical Systems Perspective
  4. Stereotypical Representations of Indian Culture/ Subcultures
  5. Methods
  6. Recruitment and Focus Group Descriptions
  7. Thematic Analysis
  8. Findings
  9. Exoticism within the Outputs of T2Is
  10. Cultural Misappropriation within the Outputs of T2Is
  11. Documenting Representational Harms within T2I Outputs
  12. Stereotyping and Erasure
  13. Disparagement and Quality of Service
  14. Disparagement and Stereotyping
  15. Dehumanization and Erasure
  16. ...and 10 more sections

Figures (9)

  • Figure 1: 2x2 T2I Outputs showing Exoticized depictions of Indian culture
  • Figure 2: 2x2 T2I Outputs showing Culturally Misappropriated depictions of Indian culture
  • Figure 3: 2x2 T2I Outputs showing Representational Harms within depictions of Indian culture
  • Figure 4: Design Recommendations based on community inputs for culturally sensitive development for T2I and GAI models.
  • Figure 5: Stable Diffusion output for the prompt 'Indian people in Western attire'.
  • ...and 4 more figures