Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diachronic and synchronic variation in the performance of adaptive machine learning systems: The ethical challenges

Joshua Hatherley, Robert Sparrow

TL;DR

The paper investigates the ethical challenges of adaptive medical AI that evolves post-deployment, foregrounding diachronic evolution and synchronic variation as key sources of risk. It argues that the traditional focus on the regulatory ‘update problem’ is insufficient and that site- and time-dependent variations complicate interpretation, consent, and equity. By outlining federated learning as a potential remedy and detailing six ethical dimensions, the authors provide a governance-oriented framework for safely realizing continual learning in clinical care. The work highlights practical trade-offs in regulation, infrastructure, and policy that must be navigated to harness the benefits of adaptive AI while protecting patients across diverse settings.

Abstract

Objectives: Machine learning (ML) has the potential to facilitate "continual learning" in medicine, in which an ML system continues to evolve in response to exposure to new data over time, even after being deployed in a clinical setting. In this paper, we provide a tutorial on the range of ethical issues raised by the use of such "adaptive" ML systems in medicine that have, thus far, been neglected in the literature. Target audience: The target audiences for this tutorial are the developers of machine learning AI systems, healthcare regulators, the broader medical informatics community, and practicing clinicians. Scope: Discussions of adaptive ML systems to date have overlooked the distinction between two sorts of variance that such systems may exhibit -- diachronic evolution (change over time) and synchronic variation (difference between cotemporaneous instantiations of the algorithm at different sites) -- and under-estimated the significance of the latter. We highlight the challenges that diachronic evolution and synchronic variation present for the quality of patient care, informed consent, and equity, and discuss the complex ethical trade-offs involved in the design of such systems.

Diachronic and synchronic variation in the performance of adaptive machine learning systems: The ethical challenges

TL;DR

The paper investigates the ethical challenges of adaptive medical AI that evolves post-deployment, foregrounding diachronic evolution and synchronic variation as key sources of risk. It argues that the traditional focus on the regulatory ‘update problem’ is insufficient and that site- and time-dependent variations complicate interpretation, consent, and equity. By outlining federated learning as a potential remedy and detailing six ethical dimensions, the authors provide a governance-oriented framework for safely realizing continual learning in clinical care. The work highlights practical trade-offs in regulation, infrastructure, and policy that must be navigated to harness the benefits of adaptive AI while protecting patients across diverse settings.

Abstract

Objectives: Machine learning (ML) has the potential to facilitate "continual learning" in medicine, in which an ML system continues to evolve in response to exposure to new data over time, even after being deployed in a clinical setting. In this paper, we provide a tutorial on the range of ethical issues raised by the use of such "adaptive" ML systems in medicine that have, thus far, been neglected in the literature. Target audience: The target audiences for this tutorial are the developers of machine learning AI systems, healthcare regulators, the broader medical informatics community, and practicing clinicians. Scope: Discussions of adaptive ML systems to date have overlooked the distinction between two sorts of variance that such systems may exhibit -- diachronic evolution (change over time) and synchronic variation (difference between cotemporaneous instantiations of the algorithm at different sites) -- and under-estimated the significance of the latter. We highlight the challenges that diachronic evolution and synchronic variation present for the quality of patient care, informed consent, and equity, and discuss the complex ethical trade-offs involved in the design of such systems.

Paper Structure

This paper contains 13 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Adaptive machine learning in medicine
  3. The update problem
  4. Two types of variation: Diachronic and synchronic
  5. Federated learning to prevent synchronic variation
  6. Ethical considerations
  7. Impact on quality of care
  8. Challenges to clinical interpretation
  9. Impact on doctor-patient relations
  10. Challenges to informed consent
  11. Equity and obsolescence
  12. Costs to particular cohorts
  13. Conclusion

Figures (1)

  • Figure 1: Schematic representation of diachronic evolution (y-axis) and synchronic variation (x-axis) in a MAMLS deployed across two sites.