Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work

Alice Qian Zhang, Judith Amores, Mary L. Gray, Mary Czerwinski, Jina Suh

TL;DR

This study investigates the nature and challenges of content work that supports RAI efforts, or "RAI content work," that spans content moderation, data labeling, and red teaming -- through the lived experiences of content workers.

Abstract

Behind the scenes of maintaining the safety of technology products from harmful and illegal digital content lies unrecognized human labor. The recent rise in the use of generative AI technologies and the accelerating demands to meet responsible AI (RAI) aims necessitates an increased focus on the labor behind such efforts in the age of AI. This study investigates the nature and challenges of content work that supports RAI efforts, or "RAI content work," that span content moderation, data labeling, and red teaming -- through the lived experiences of content workers. We conduct a formative survey and semi-structured interview studies to develop a conceptualization of RAI content work and a subsequent framework of recommendations for providing holistic support for content workers. We validate our recommendations through a series of workshops with content workers and derive considerations for and examples of implementing such recommendations. We discuss how our framework may guide future innovation to support the well-being and professional development of the RAI content workforce.

AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work

TL;DR

This study investigates the nature and challenges of content work that supports RAI efforts, or "RAI content work," that spans content moderation, data labeling, and red teaming -- through the lived experiences of content workers.

Abstract

Behind the scenes of maintaining the safety of technology products from harmful and illegal digital content lies unrecognized human labor. The recent rise in the use of generative AI technologies and the accelerating demands to meet responsible AI (RAI) aims necessitates an increased focus on the labor behind such efforts in the age of AI. This study investigates the nature and challenges of content work that supports RAI efforts, or "RAI content work," that span content moderation, data labeling, and red teaming -- through the lived experiences of content workers. We conduct a formative survey and semi-structured interview studies to develop a conceptualization of RAI content work and a subsequent framework of recommendations for providing holistic support for content workers. We validate our recommendations through a series of workshops with content workers and derive considerations for and examples of implementing such recommendations. We discuss how our framework may guide future innovation to support the well-being and professional development of the RAI content workforce.

Paper Structure

This paper contains 40 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conceptualizing Content Work for Responsible AI
  4. Well-being Challenges for Content Workers from Harmful Content Exposure
  5. Workplace Support for Well-being and Harmful Content Exposure
  6. Phase 1: Understanding the Nature and Challenges of Content Work
  7. Methods
  8. Survey Study
  9. Interview Study
  10. Survey and Interview Data Analysis
  11. Privacy and Positionality Statement
  12. Survey and Interview Findings
  13. RQ1: What is the nature of content work?
  14. How content workers define their profession
  15. Types and modalities of content that workers engage with
  16. ...and 25 more sections

Figures (4)

  • Figure 1: Flow of our two-phase study. In the first phase, we conducted a survey study (N=67) and an interview study (N=22) to understand the nature of content work. From these insights, we developed a set of recommendations to improve content worker well-being. In the second phase, we validated the challenges we discovered and our recommendations to address those challenges, within the AURA framework that organize those recommendations, through interactive workshops (N=14).
  • Figure 2: Percentages of our survey population exposed to various content types (e.g., text, images, audio) and categories (e.g., hate speech, self-harm) across everyone (N=67), content moderation (N=51), data labeling (N=36), and red teaming (N=9).
  • Figure 3: The access and usefulness of tools that help during content work which are sorted in descending order of usefulness with 95% confidence intervals.
  • Figure 4: The access and usefulness of coping strategies used to manage the demands of content work which are sorted in descending order of usefulness with 95% confidence intervals.