Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act

Lucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding, Gerasimos Spanakis, Gijs van Dijck, Konrad Kollnig

Abstract

Technical and legal debates frequently suggest that "accuracy" is an objective, measurable, and purely technical property. We challenge this view, showing that evaluating AI performance fundamentally depends on context-dependent normative decisions. These techno-normative choices are crucial for rigorous AI deployment, as they determine which errors are prioritised, how risks are distributed, and how trade-offs between competing objectives are resolved. This paper provides a legal-technical analysis of the choices that shape how accuracy is defined, measured, and assessed, using the 2024 European Union AI Act -- which mandates an "appropriate level of accuracy" for high-risk systems -- as a primary case study. We identify and analyse four choices central to any robust performance evaluation: (1) selecting metrics, (2) balancing multiple metrics, (3) measuring metrics against representative data, and (4) determining acceptance thresholds. For each choice, we study its relationship to the AI Act's accuracy requirement and associated documentation obligations, show how its technical implementation embeds implicit or explicit assumptions about acceptable risks, errors, and trade-offs, and discuss the implications for the practical implementation of the AI Act by examples and related technical standards. By making the techno-normative dimensions of accuracy explicit, this paper contributes to broader interdisciplinary debates on AI governance and regulation, and offers specific guidance for regulators, auditors, and developers tasked with translating (legal) safety requirements into technical practice.

Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act

Abstract

Technical and legal debates frequently suggest that "accuracy" is an objective, measurable, and purely technical property. We challenge this view, showing that evaluating AI performance fundamentally depends on context-dependent normative decisions. These techno-normative choices are crucial for rigorous AI deployment, as they determine which errors are prioritised, how risks are distributed, and how trade-offs between competing objectives are resolved. This paper provides a legal-technical analysis of the choices that shape how accuracy is defined, measured, and assessed, using the 2024 European Union AI Act -- which mandates an "appropriate level of accuracy" for high-risk systems -- as a primary case study. We identify and analyse four choices central to any robust performance evaluation: (1) selecting metrics, (2) balancing multiple metrics, (3) measuring metrics against representative data, and (4) determining acceptance thresholds. For each choice, we study its relationship to the AI Act's accuracy requirement and associated documentation obligations, show how its technical implementation embeds implicit or explicit assumptions about acceptable risks, errors, and trade-offs, and discuss the implications for the practical implementation of the AI Act by examples and related technical standards. By making the techno-normative dimensions of accuracy explicit, this paper contributes to broader interdisciplinary debates on AI governance and regulation, and offers specific guidance for regulators, auditors, and developers tasked with translating (legal) safety requirements into technical practice.

Paper Structure

This paper contains 29 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Accuracy in the EU AI Act
  3. The AI Act's High-Risk Requirements, including accuracy
  4. The Function of the Accuracy Obligation
  5. Ongoing Efforts Towards Operationalising the Accuracy Obligation: Technical Standards and EU Benchmarks
  6. Harmonised standards
  7. Benchmarks and measurement methodologies
  8. From Legal Requirements to Technical Practice
  9. Accuracy and performance evaluations in ML workflows
  10. Structure of the Analysis and Illustrative Use Case
  11. Choice 1: Selecting Metrics
  12. The choice and its relevance for the AI Act
  13. Technical considerations: Metric selection in practice
  14. Class imbalanced datasets.
  15. Asymmetric error costs.
  16. ...and 14 more sections

Figures (1)

  • Figure 1: Illustration of the four techno-normative choices of performance evaluations, as examined in this paper. Depending on (1) the chosen metric(s), (2) how they are balanced, (3) how they are measured, and (4) the way in which the acceptance region is defined, different models may be considered more or less accurate. These four choices are thus both central to performance evaluations and directly relevant to the AI Act's requirement for high-risk AI systems to "achieve an appropriate level of accuracy".