Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safeguarding Autonomy: a Focus on Machine Learning Decision Systems

Paula Subías-Beltrán, Oriol Pujol, Itziar de Lecuona

TL;DR

The paper tackles the risky gap between autonomy theory and ML practice by proposing a diagnostic framework that audits autonomy across the ML pipeline. It identifies four key vigilance factors—competence development, over-reliance, heteronomy, and misalignment—and provides actionable questions aligned with each pipeline stage to guide practitioners and ML auditors. By mapping philosophical concepts of agency, authenticity, and authority to concrete ML phases (world, knowledge representation, model, prediction), the framework aims to promote transparency, accountability, and end-user empowerment. This work offers a practical tool for responsible AI development and aligns with regulatory aims to safeguard autonomy in decision-making systems.

Abstract

As global discourse on AI regulation gains momentum, this paper focuses on delineating the impact of ML on autonomy and fostering awareness. Respect for autonomy is a basic principle in bioethics that establishes persons as decision-makers. While the concept of autonomy in the context of ML appears in several European normative publications, it remains a theoretical concept that has yet to be widely accepted in ML practice. Our contribution is to bridge the theoretical and practical gap by encouraging the practical application of autonomy in decision-making within ML practice by identifying the conditioning factors that currently prevent it. Consequently, we focus on the different stages of the ML pipeline to identify the potential effects on ML end-users' autonomy. To improve its practical utility, we propose a related question for each detected impact, offering guidance for identifying possible focus points to respect ML end-users autonomy in decision-making.

Safeguarding Autonomy: a Focus on Machine Learning Decision Systems

TL;DR

The paper tackles the risky gap between autonomy theory and ML practice by proposing a diagnostic framework that audits autonomy across the ML pipeline. It identifies four key vigilance factors—competence development, over-reliance, heteronomy, and misalignment—and provides actionable questions aligned with each pipeline stage to guide practitioners and ML auditors. By mapping philosophical concepts of agency, authenticity, and authority to concrete ML phases (world, knowledge representation, model, prediction), the framework aims to promote transparency, accountability, and end-user empowerment. This work offers a practical tool for responsible AI development and aligns with regulatory aims to safeguard autonomy in decision-making systems.

Abstract

As global discourse on AI regulation gains momentum, this paper focuses on delineating the impact of ML on autonomy and fostering awareness. Respect for autonomy is a basic principle in bioethics that establishes persons as decision-makers. While the concept of autonomy in the context of ML appears in several European normative publications, it remains a theoretical concept that has yet to be widely accepted in ML practice. Our contribution is to bridge the theoretical and practical gap by encouraging the practical application of autonomy in decision-making within ML practice by identifying the conditioning factors that currently prevent it. Consequently, we focus on the different stages of the ML pipeline to identify the potential effects on ML end-users' autonomy. To improve its practical utility, we propose a related question for each detected impact, offering guidance for identifying possible focus points to respect ML end-users autonomy in decision-making.

Paper Structure

This paper contains 15 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Autonomy and decision-making in ML
  3. On decision-making
  4. How can ML impact autonomy on decision-making?
  5. Fail to develop competence of autonomy
  6. Over-reliance
  7. Heteronomy dynamics
  8. Misalignment
  9. Dissecting the effect on autonomy in the ML pipeline
  10. Impact on the world
  11. Impact on knowledge representation
  12. Impact on the ML model
  13. Impact on the prediction
  14. Conclusions
  15. Acknowledgements

Figures (2)

  • Figure 1: A simplified version of the ML pipeline.
  • Figure 2: Framework for investigating the impact on autonomy via the lens of the ML pipeline. The area of vigilance is indicated based on color and marker shape.