The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing

TL;DR

Rapid growth of general-purpose AI has amplified the need to stress-test systems through red teaming, raising questions about the human factors involved. This paper proposes a one-day hybrid CSCW workshop to study AI red-teaming practices, drawing on historical perspectives and HCI/CSCW literature to center the human workers and labor conditions. It outlines themes on conceptualization, labor dynamics, and well-being, plus actionable workshop activities (red-teaming exercises, panels, artifacts) intended to yield a research agenda and practical toolkit. The expected outcome is an interdisciplinary AI red-teaming research network and a synthesized report that can guide practitioners and researchers in mitigating harms while supporting red-teamers.

Abstract

Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any have investigated red teaming itself. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection.