Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems

Djalel Benbouzid, Christiane Plociennik, Laura Lucaj, Mihai Maftei, Iris Merget, Aljoscha Burchardt, Marc P. Hauer, Abdeldjallil Naceri, Patrick van der Smagt

TL;DR

The paper tackles the gap between abstract ethical guidelines and practical governance of ML systems by proposing a pragmatic, lifecycle-based auditing procedure anchored to EC ALTAI risk assessment. It combines a common lifecycle model with an ISACA-inspired audit framework to scope, document, and test ML deployments, demonstrated through two real-world pilots. Key contributions include a reusable lifecycle mapping, ALTAI-driven risk assessment, and guidance on evidence collection, testing, and continuous auditing, along with lessons on auditability criteria and the need for a shared risk database. The approach promises scalable, transparent, and accountable ML auditing suitable for internal and external audits and informs ongoing standards and regulatory efforts.

Abstract

The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transparency and accountability, and a principled risk assessment procedure that allows the proper scoping of the audit. Aiming to make a pragmatic step towards a wider adoption of ML auditing, we present a respective procedure that extends the AI-HLEG guidelines published by the European Commission. Our audit procedure is based on an ML lifecycle model that explicitly focuses on documentation, accountability, and quality assurance; and serves as a common ground for alignment between the auditors and the audited organisation. We describe two pilots conducted on real-world use cases from two different organisations and discuss the shortcomings of ML algorithmic auditing as well as future directions thereof.

Pragmatic auditing: a pilot-driven approach for auditing Machine Learning systems

TL;DR

The paper tackles the gap between abstract ethical guidelines and practical governance of ML systems by proposing a pragmatic, lifecycle-based auditing procedure anchored to EC ALTAI risk assessment. It combines a common lifecycle model with an ISACA-inspired audit framework to scope, document, and test ML deployments, demonstrated through two real-world pilots. Key contributions include a reusable lifecycle mapping, ALTAI-driven risk assessment, and guidance on evidence collection, testing, and continuous auditing, along with lessons on auditability criteria and the need for a shared risk database. The approach promises scalable, transparent, and accountable ML auditing suitable for internal and external audits and informs ongoing standards and regulatory efforts.

Abstract

The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A lifecycle model that is tailored towards transparency and accountability, and a principled risk assessment procedure that allows the proper scoping of the audit. Aiming to make a pragmatic step towards a wider adoption of ML auditing, we present a respective procedure that extends the AI-HLEG guidelines published by the European Commission. Our audit procedure is based on an ML lifecycle model that explicitly focuses on documentation, accountability, and quality assurance; and serves as a common ground for alignment between the auditors and the audited organisation. We describe two pilots conducted on real-world use cases from two different organisations and discuss the shortcomings of ML algorithmic auditing as well as future directions thereof.
Paper Structure (51 sections, 7 figures)

This paper contains 51 sections, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Literature on algorithmic auditing
  4. Toolkits
  5. Guidelines and regulation in the EU
  6. Discussion
  7. The audit procedure
  8. The planning phase
  9. Lifecycle model mapping
  10. Risk assessment
  11. The fieldwork and documentation phase
  12. Collection of evidence
  13. Compliance testing
  14. Custom testing
  15. The reporting phase
  16. ...and 36 more sections

Figures (7)

  • Figure 1: A high-level view of the audit process.
  • Figure 2: A lifecycle model for ML systems design to align the auditor with the auditing organisation. Each of the steps can be subject of a required documentation, a proper versioning, and a clear definition of ownership and roles. Furthermore, the lifecycle model is accompanied by a risk-assessment procedure that relates to each of its steps.
  • Figure 3: The focus of the first pilot. For each of the blue steps, a thorough inspection of the documentation, roles, and ownership was conducted. These steps also served narrow the scope of the risk-assessment. The yellow steps were highlighted as expected but missing.
  • Figure 4: GARMI: service humanoid robot Garmi_2021. (a) Description of GARMI's different modules. (b) GARMI helping an elderly person in rehabilitation scenario.
  • Figure 5: The focus of the second pilot. For each of the blue steps, a thorough inspection of the documentation, roles, and ownership was conducted. These steps also served narrow the scope of the risk assessment. The yellow steps were highlighted as expected but missing.
  • ...and 2 more figures