Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Provocation: Who benefits from "inclusion" in Generative AI?

Samantha Dalal, Siobhan Mackenzie Hall, Nari Johnson

TL;DR

It is argued that dominant structures of community participation in AI development and evaluation are not explicit enough about the benefits and harms that members of socially marginalized groups may experience as a result of their participation.

Abstract

The demands for accurate and representative generative AI systems means there is an increased demand on participatory evaluation structures. While these participatory structures are paramount to to ensure non-dominant values, knowledge and material culture are also reflected in AI models and the media they generate, we argue that dominant structures of community participation in AI development and evaluation are not explicit enough about the benefits and harms that members of socially marginalized groups may experience as a result of their participation. Without explicit interrogation of these benefits by AI developers, as a community we may remain blind to the immensity of systemic change that is needed as well. To support this provocation, we present a speculative case study, developed from our own collective experiences as AI researchers. We use this speculative context to itemize the barriers that need to be overcome in order for the proposed benefits to marginalized communities to be realized, and harms mitigated.

Provocation: Who benefits from "inclusion" in Generative AI?

TL;DR

It is argued that dominant structures of community participation in AI development and evaluation are not explicit enough about the benefits and harms that members of socially marginalized groups may experience as a result of their participation.

Abstract

The demands for accurate and representative generative AI systems means there is an increased demand on participatory evaluation structures. While these participatory structures are paramount to to ensure non-dominant values, knowledge and material culture are also reflected in AI models and the media they generate, we argue that dominant structures of community participation in AI development and evaluation are not explicit enough about the benefits and harms that members of socially marginalized groups may experience as a result of their participation. Without explicit interrogation of these benefits by AI developers, as a community we may remain blind to the immensity of systemic change that is needed as well. To support this provocation, we present a speculative case study, developed from our own collective experiences as AI researchers. We use this speculative context to itemize the barriers that need to be overcome in order for the proposed benefits to marginalized communities to be realized, and harms mitigated.

Paper Structure

This paper contains 4 sections, 1 figure.

Table of Contents

  1. Speculative Case Study: How do community members realize benefits & harms from improved GenAI performance?
  2. Broader impact statement
  3. Limitations
  4. Imagining paths forward

Figures (1)

  • Figure 1: We map the dependencies that often exist between key stakeholder groups: (1) members of marginalized communities, (2) other social actors who use GenAI tools, and (3) institutions (such as private technology companies or academic universities) that develop GenAI models, in participatory engagements to improve how marginalized communities are represented by GenAI. In this paper, we discuss how these dependencies pose barriers for marginalized communities to realize the benefits of improved AI models.