Do Responsible AI Artifacts Advance Stakeholder Goals? Four Key Barriers Perceived by Legal and Civil Stakeholders

TL;DR

The paper examines whether Multi-Actor Responsible AI Artifacts effectively advance stakeholder goals by eliciting the perspectives of legal/regulatory and civil society actors. Using 19 in-situ stakeholders and three design activities, the study uncovers four barriers—end-users as a second-order priority, selective showcasing of laudable models, over-reliance on transparency, and offloading responsibility to end-users—that may hinder external oversight and governance. It discusses how misaligned goals between industry and non-industry actors shape artifact design and usage, and offers design and policy implications to foster more collaborative, proactive cross-sector governance. The work contributes empirical insights into artifact design for governance beyond industry practitioners and highlights pathways to restructure incentives, disclosure practices, and stakeholder engagement to strengthen downstream harm protection.

Abstract

The responsible AI (RAI) community has introduced numerous processes and artifacts (e.g., Model Cards, Transparency Notes, Data Cards) to facilitate transparency and support the governance of AI systems. While originally designed to scaffold and document AI development processes in technology companies, these artifacts are becoming central components of regulatory compliance under recent regulations such as the EU AI Act. Much prior work has explored the design of new RAI artifacts or their use by practitioners within technology companies. However, as RAI artifacts begin to play key roles in enabling external oversight, it becomes critical to understand how stakeholders--particularly those situated outside of technology companies who govern and audit industry AI deployments--perceive the efficacy of RAI artifacts. In this study, we conduct semi-structured interviews and design activities with 19 government, legal, and civil society stakeholders who inform policy and advocacy around responsible AI efforts. While participants believe that RAI artifacts are a valuable contribution to the broader AI governance ecosystem, many are concerned about their potential unintended, longer-term impacts on actors outside of technology companies (e.g., downstream end-users, policymakers, civil society stakeholders). We organize these beliefs into four barriers that help explain how RAI artifacts may (inadvertently) reconfigure power relations across civil society, government, and industry, impeding civil society and legal stakeholders' ability to protect downstream end-users from potential AI harms. Participants envision how structural changes, along with changes in how RAI artifacts are designed, used, and governed, could help redirect the role of artifacts to support more collaborative and proactive external oversight of AI systems. We discuss research and policy implications for RAI artifacts.

Figures (3)

• Figure 1: Overview of the three activities in our study protocol
• Figure 2: Participants described how (1) the technology industry exists within a multi-actor RAI ecosystem, which shape their goals and incentives for RAI governance. These goals and incentives (2) shape how the technology industry designs and uses Multi-Actor RAI Artifact; these decisions have rippling impacts on other actors within the RAI ecosystem. A more detailed breakdown of the properties and impacts of this dynamic are described in Figure 3.
• Figure 3: A summary of the four barriers to Multi-Actor RAI Artifacts surfaced by participants, each mapping to one subsection of the findings. For each barrier, we describe (1) Factors: the incentives, goals, and theories of change of the technology industry that contribute to the barrier, (2) Signals: ways in which the barriers become visible through design and use practices around RAI artifacts, and (3) Impacts: the broader downstream impacts the barriers have on the responsible AI ecosystem and society.