Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil society

TL;DR

This paper analyzes the EU regulatory landscape (DSA and AIA) for AI auditing, arguing that current frameworks create a regulatory gap that limits civil society and academic oversight. It defines an AI audit ecosystem that emphasizes third-party audits by researchers and civil society, and it highlights the need for data and model access to enable meaningful scrutiny beyond internal assessments. By reviewing provisions and case studies, the authors propose concrete policy recommendations, including extending data access to vetted researchers, including NGOs and journalists, and incentivizing and publishing audit results to foster accountability. The work underscores the practical impact of robust, inclusive auditing on preventing harm from AI systems and on strengthening democratic oversight of AI deployment across the EU.

Abstract

The European legislature has proposed the Digital Services Act (DSA) and Artificial Intelligence Act (AIA) to regulate platforms and Artificial Intelligence (AI) products. We review to what extent third-party audits are part of both laws and how is access to information on models and the data provided. By considering the value of third-party audits and third-party data access in an audit ecosystem, we identify a regulatory gap in that the AIA does not provide access to data for researchers and civil society. Our contributions to the literature include: (1) Defining an AI audit ecosystem incorporating compliance and oversight. (2) Highlighting a regulatory gap within the DSA and AIA regulatory framework, preventing the establishment of an AI audit ecosystem that has effective oversight by civil society and academia. (3) Emphasizing that third-party audits by research and civil society must be part of that ecosystem, we call for AIA amendments and delegated acts to include data and model access for certain AI products. Furthermore, we call for the DSA to provide NGOs and investigative journalists with data access to platforms by delegated acts and for adaptions and amendments of the AIA to provide third-party audits and data and model access, at least for high-risk systems. Regulations modeled after EU AI regulations should enable data access and third-party audits, fostering an AI audit ecosystem that promotes compliance and oversight mechanisms.