Power and Play: Investigating "License to Critique" in Teams' AI Ethics Discussions

TL;DR

The paper investigates how power dynamics and organizational norms shape the license to critique in AI ethics discussions and evaluates whether a speculative game can expand discussion without constraining critique. Using the What Could Go Wrong? game with four teams (three corporate, one activist) and follow-up interviews, the authors identify scope, time pressures, and role-based compartmentalization as key drivers of discursive closure, while manager presence and perceived credibility further modulate willingness to speak up. The findings suggest that game-based interventions can broaden conversation and reveal allies, but are unlikely to induce direct changes in products or practices due to existing power structures and the gap between hypothetical contexts and real-world action. The study contributes to AI-ethics methods by highlighting the importance of power-aware design in interventions and by reframing games as tools for ally-building and cross-understanding rather than guaranteed pathways to policy or product changes.

Abstract

Past work has sought to design AI ethics interventions--such as checklists or toolkits--to help practitioners design more ethical AI systems. However, other work demonstrates how these interventions may instead serve to limit critique to that addressed within the intervention, while rendering broader concerns illegitimate. In this paper, drawing on work examining how standards enact discursive closure and how power relations affect whether and how people raise critique, we recruit three corporate teams, and one activist team, each with prior context working with one another, to play a game designed to trigger broad discussion around AI ethics. We use this as a point of contrast to trigger reflection on their teams' past discussions, examining factors which may affect their "license to critique" in AI ethics discussions. We then report on how particular affordances of this game may influence discussion, and find that the hypothetical context created in the game is unlikely to be a viable mechanism for real world change. We discuss how power dynamics within a group and notions of "scope" affect whether people may be willing to raise critique in AI ethics discussions, and discuss our finding that games are unlikely to enable direct changes to products or practice, but may be more likely to allow members to find critically-aligned allies for future collective action.