Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"Detective Work We Shouldn't Have to Do": Practitioner Challenges in Regulatory-Aligned Data Quality in Machine Learning Systems

Yichun Wang, Kristina Irion, Paul Groth, Hazar Harmouch

TL;DR

This paper investigates how EU data practitioners align data quality in machine learning with GDPR and AI Act requirements. Using a qualitative, interview-based study of 14 practitioners across regulated domains, it reveals that while awareness of regulatory demands is high, translating principles into concrete engineering practices is impeded by fragmented data pipelines, tool limitations, and unclear ownership between technical and legal teams. The authors identify key challenges—traceability gaps, difficulties propagating deletion across data lifecycles, and reliance on ad-hoc documentation and manual processes—and propose practitioner-informed directions for compliance-aware tooling, governance structures, and proactive data governance cultures. The findings argue for a socio-technical reframing of data quality that integrates legal context into ML infrastructure and processes, enabling more reliable, auditable, and compliant AI systems with reduced audit and regulatory risk.

Abstract

Ensuring data quality in machine learning (ML) systems has become increasingly complex as regulatory requirements expand. In the European Union (EU), frameworks such as the General Data Protection Regulation (GDPR) and the Artificial Intelligence Act (AI Act) articulate data quality requirements that closely parallel technical concerns in ML practice, while also extending to legal obligations related to accountability, risk management, and human rights protection. This paper presents a qualitative interview study with EU-based data practitioners working on ML systems in regulated contexts. Through semi-structured interviews, we investigate how practitioners interpret regulatory-aligned data quality, the challenges they encounter, and the supports they identify as necessary. Our findings reveal persistent gaps between legal principles and engineering workflows, fragmentation across data pipelines, limitations of existing tools, unclear responsibility boundaries between technical and legal teams, and a tendency toward reactive, audit-driven quality practices. We also identify practitioners' needs for compliance-aware tooling, clearer governance structures, and cultural shifts toward proactive data governance.

"Detective Work We Shouldn't Have to Do": Practitioner Challenges in Regulatory-Aligned Data Quality in Machine Learning Systems

TL;DR

This paper investigates how EU data practitioners align data quality in machine learning with GDPR and AI Act requirements. Using a qualitative, interview-based study of 14 practitioners across regulated domains, it reveals that while awareness of regulatory demands is high, translating principles into concrete engineering practices is impeded by fragmented data pipelines, tool limitations, and unclear ownership between technical and legal teams. The authors identify key challenges—traceability gaps, difficulties propagating deletion across data lifecycles, and reliance on ad-hoc documentation and manual processes—and propose practitioner-informed directions for compliance-aware tooling, governance structures, and proactive data governance cultures. The findings argue for a socio-technical reframing of data quality that integrates legal context into ML infrastructure and processes, enabling more reliable, auditable, and compliant AI systems with reduced audit and regulatory risk.

Abstract

Ensuring data quality in machine learning (ML) systems has become increasingly complex as regulatory requirements expand. In the European Union (EU), frameworks such as the General Data Protection Regulation (GDPR) and the Artificial Intelligence Act (AI Act) articulate data quality requirements that closely parallel technical concerns in ML practice, while also extending to legal obligations related to accountability, risk management, and human rights protection. This paper presents a qualitative interview study with EU-based data practitioners working on ML systems in regulated contexts. Through semi-structured interviews, we investigate how practitioners interpret regulatory-aligned data quality, the challenges they encounter, and the supports they identify as necessary. Our findings reveal persistent gaps between legal principles and engineering workflows, fragmentation across data pipelines, limitations of existing tools, unclear responsibility boundaries between technical and legal teams, and a tendency toward reactive, audit-driven quality practices. We also identify practitioners' needs for compliance-aware tooling, clearer governance structures, and cultural shifts toward proactive data governance.
Paper Structure (22 sections, 2 figures, 2 tables)

This paper contains 22 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature review
  3. Data quality & ML: the technical perspective
  4. Regulatory perspectives on data quality
  5. Empirical studies on practitioner experience and organisational dynamics
  6. Research gap
  7. Methodology
  8. Interview design
  9. Recruitment and Implementation
  10. Analysis
  11. Limitations
  12. Interview Results
  13. Real-world challenges in ensuring regulatory-aligned data quality
  14. Supports practitioners need to manage regulatory-aligned data quality
  15. Discussion and Recommendations
  16. ...and 7 more sections

Figures (2)

  • Figure 1: Regulatory-Aligned Data Quality in Practice - Challenges and Desiderata
  • Figure 2: Interview outline used in the semi-structured interviews. The outline guided discussions on data quality practices, compliance-related challenges, and regulatory-aligned trade-offs.