Doing Audits Right? The Role of Sampling and Legal Content Analysis in Systemic Risk Assessments and Independent Audits in the Digital Services Act

TL;DR

This paper addresses how to conduct evidence-based audits under the EU Digital Services Act by assessing systemic risks through a mixed-method approach that integrates statistical content sampling with legal content analysis. It maps the DR-driven sampling requirements to practical auditing workflows for Very Large Online Platforms and Search Engines, and compares 19 platform and audit reports to reveal substantial variation in methodology and transparency. The authors argue for a standardized yet flexible framework that combines ex ante and ex post sampling with codified legal analysis, enabling more robust risk assessment, better comparability across audits, and actionable insights for regulators and platforms. The work contributes to DSA compliance research by proposing concrete methodological structures, governance mechanisms (e.g., a Moderation Decision Supervisory Team), and documentation practices to enhance accountability, fairness, and transparency in digital governance.

Abstract

A central requirement of the European Union's Digital Services Act (DSA) is that online platforms undergo internal and external audits. A key component of these audits is the assessment of systemic risks, including the dissemination of illegal content, threats to fundamental rights, impacts on democratic processes, and gender-based violence. The DSA Delegated Regulation outlines how such audits should be conducted, setting expectations for both platforms and auditors. This article evaluates the strengths and limitations of different qualitative and quantitative methods for auditing these systemic risks and proposes a mixed-method approach for DSA compliance. We argue that content sampling, combined with legal and empirical analysis, offers a viable method for risk-specific audits. First, we examine relevant legal provisions on sample selection for audit purposes. We then assess sampling techniques and methods suitable for detecting systemic risks, focusing on how representativeness can be understood across disciplines. Finally, we review initial systemic risk assessment reports submitted by platforms, analyzing their testing and sampling methodologies. By proposing a structured, mixed-method approach tailored to specific risk categories and platform characteristics, this article addresses the challenge of evidence-based audits under the DSA. Our contribution emphasizes the need for adaptable, context-sensitive auditing strategies and adds to the emerging field of DSA compliance research.