It's complicated. The relationship of algorithmic fairness and non-discrimination regulations for high-risk systems in the EU AI Act

TL;DR

The paper analyzes how EU non-discrimination regulation interacts with algorithmic fairness within the EU AI Act, focusing on high-risk systems. It provides a two-part analysis: a primer on legal and fairness concepts, and a detailed examination of the Act’s provisions, risk management, and standardisation processes. It identifies three key findings: most non-discrimination rules target high-risk systems; regulation covers input data biases and monitoring of outputs but exhibits inconsistencies and feasibility concerns; and there is a need for future work to align Act provisions with traditional non-discrimination law and develop robust auditing methodologies. The work aims to lay the groundwork for interdisciplinary collaboration and more precise standardisation in auditing fairness for EU AI regulation.

Abstract

What constitutes a fair decision? This question is not only difficult for humans but becomes more challenging when Artificial Intelligence (AI) models are used. In light of discriminatory algorithmic behaviors, the EU has recently passed the AI Act, which mandates specific rules for high-risk systems, incorporating both traditional legal non-discrimination regulations and machine learning based algorithmic fairness concepts. This paper aims to bridge these two different concepts in the AI Act through: First, a necessary high-level introduction of both concepts targeting legal and computer science-oriented scholars, and second, an in-depth analysis of the AI Act's relationship between legal non-discrimination regulations and algorithmic fairness. Our analysis reveals three key findings: (1.) Most non-discrimination regulations target only high-risk AI systems. (2.) The regulation of high-risk systems encompasses both data input requirements and output monitoring, though these regulations are partly inconsistent and raise questions of computational feasibility. (3.) Finally, we consider the possible (future) interaction of classical EU non-discrimination law and the AI Act regulations. We recommend developing more specific auditing and testing methodologies for AI systems. This paper aims to serve as a foundation for future interdisciplinary collaboration between legal scholars and computer science-oriented machine learning researchers studying discrimination in AI systems.