Ask What Your Country Can Do For You: Towards a Public Red Teaming Model

TL;DR

The paper addresses the responsibility gap in AI safety oversight by proposing cooperative public red-teaming as a TEVV-aligned approach to adversarial evaluation. It analyzes three pilot exercises—NIST ARIA, a CAMLIS 2024 demonstrator, and IMDA Singapore—highlighting how public, third-party participation expands harm-surface coverage, benchmarks, and governance insights across jurisdictions. The findings suggest that such public red-teaming can yield meaningful, scalable results that inform safety standards, benchmarks, and regulatory practices, while engaging diverse stakeholders including SMEs and civil society. Overall, the approach offers a practical path toward responsible AI governance and broader public-interest oversight in AI development and deployment.

Abstract

AI systems have the potential to produce both benefits and harms, but without rigorous and ongoing adversarial evaluation, AI actors will struggle to assess the breadth and magnitude of the AI risk surface. Researchers from the field of systems design have developed several effective sociotechnical AI evaluation and red teaming techniques targeting bias, hate speech, mis/disinformation, and other documented harm classes. However, as increasingly sophisticated AI systems are released into high-stakes sectors (such as education, healthcare, and intelligence-gathering), our current evaluation and monitoring methods are proving less and less capable of delivering effective oversight. In order to actually deliver responsible AI and to ensure AI's harms are fully understood and its security vulnerabilities mitigated, pioneering new approaches to close this "responsibility gap" are now more urgent than ever. In this paper, we propose one such approach, the cooperative public AI red-teaming exercise, and discuss early results of its prior pilot implementations. This approach is intertwined with CAMLIS itself: the first in-person public demonstrator exercise was held in conjunction with CAMLIS 2024. We review the operational design and results of this exercise, the prior National Institute of Standards and Technology (NIST)'s Assessing the Risks and Impacts of AI (ARIA) pilot exercise, and another similar exercise conducted with the Singapore Infocomm Media Development Authority (IMDA). Ultimately, we argue that this approach is both capable of delivering meaningful results and is also scalable to many AI developing jurisdictions.