Lost in Vagueness: Towards Context-Sensitive Standards for Robustness Assessment under the EU AI Act

TL;DR

The paper addresses the challenge of operationalizing robustness under the EU AI Act, arguing that robustness is inherently context-dependent and must be defined through 'robustness of what' and 'robustness to what' across use case, data, and model. It proposes a context-sensitive, multi-layered standardisation framework that couples horizontal standards with domain-specific provisions, a dynamic repository of best practices, and external validation to reduce interpretative ambiguity and keep pace with technological change. Key contributions include a domain-specific standardisation approach, a context-sensitive perturbation taxonomy, and a dynamic, collaborative repository to share benchmarks, sandboxes, and lessons learned. The framework aims to enable practical, adaptable robustness assessments that support compliant, trustworthy deployment of high-risk AI systems across diverse domains.

Abstract

Robustness is a key requirement for high-risk AI systems under the EU Artificial Intelligence Act (AI Act). However, both its definition and assessment methods remain underspecified, leaving providers with little concrete direction on how to demonstrate compliance. This stems from the Act's horizontal approach, which establishes general obligations applicable across all AI systems, but leaves the task of providing technical guidance to harmonised standards. This paper investigates what it means for AI systems to be robust and illustrates the need for context-sensitive standardisation. We argue that robustness is not a fixed property of a system, but depends on which aspects of performance are expected to remain stable ("robustness of what"), the perturbations the system must withstand ("robustness to what") and the operational environment. We identify three contextual drivers--use case, data and model--that shape the relevant perturbations and influence the choice of tests, metrics and benchmarks used to evaluate robustness. The need to provide at least a range of technical options that providers can assess and implement in light of the system's purpose is explicitly recognised by the standardisation request for the AI Act, but planned standards, still focused on horizontal coverage, do not yet offer this level of detail. Building on this, we propose a context-sensitive multi-layered standardisation framework where horizontal standards set common principles and terminology, while domain-specific ones identify risks across the AI lifecycle and guide appropriate practices, organised in a dynamic repository where providers can propose new informative methods and share lessons learned. Such a system reduces the interpretative burden, mitigates arbitrariness and addresses the obsolescence of static standards, ensuring that robustness assessment is both adaptable and operationally meaningful.

Figures (2)

• Figure 1: Conceptual pipeline for context-sensitive robustness assessment.
• Figure 2: Multi-layered framework for context-sensitive robustness evaluation under the AI Act.