Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Position: A Call to Action for a Human-Centered AutoML Paradigm

Marius Lindauer, Florian Karl, Anne Klier, Julia Moosbauer, Alexander Tornede, Andreas Mueller, Frank Hutter, Matthias Feurer, Bernd Bischl

TL;DR

The paper contends that AutoML progress has predominantly targeted predictive performance, neglecting human factors across diverse user groups. It proposes a human-centered AutoML paradigm that integrates transparency, customizability, iterative interaction, and expert collaboration within the CRISP-ML(Q) lifecycle. Five hypotheses address interpretability, customization, workflow integration, expert collaboration, and user empowerment, guiding future research. The authors review existing trust and HCI work and propose modular toolkits and LLM-based interfaces to bridge gaps between algorithmic and human-centric AutoML. Ultimately, the aim is to democratize AutoML safely and effectively by combining human expertise with automated optimization.

Abstract

Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience. Over the past decade, commendable achievements in AutoML have primarily focused on optimizing predictive performance. This focused progress, while substantial, raises questions about how well AutoML has met its broader, original goals. In this position paper, we argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems, including their diverse roles, expectations, and expertise. We envision a more human-centered approach in future AutoML research, promoting the collaborative design of ML systems that tightly integrates the complementary strengths of human expertise and AutoML methodologies.

Position: A Call to Action for a Human-Centered AutoML Paradigm

TL;DR

The paper contends that AutoML progress has predominantly targeted predictive performance, neglecting human factors across diverse user groups. It proposes a human-centered AutoML paradigm that integrates transparency, customizability, iterative interaction, and expert collaboration within the CRISP-ML(Q) lifecycle. Five hypotheses address interpretability, customization, workflow integration, expert collaboration, and user empowerment, guiding future research. The authors review existing trust and HCI work and propose modular toolkits and LLM-based interfaces to bridge gaps between algorithmic and human-centric AutoML. Ultimately, the aim is to democratize AutoML safely and effectively by combining human expertise with automated optimization.

Abstract

Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience. Over the past decade, commendable achievements in AutoML have primarily focused on optimizing predictive performance. This focused progress, while substantial, raises questions about how well AutoML has met its broader, original goals. In this position paper, we argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems, including their diverse roles, expectations, and expertise. We envision a more human-centered approach in future AutoML research, promoting the collaborative design of ML systems that tightly integrates the complementary strengths of human expertise and AutoML methodologies.
Paper Structure (20 sections, 2 figures)

This paper contains 20 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Human-Centered Machine Learning
  4. Human-Centered AutoML
  5. The Case for Human-Centered AutoML
  6. Hypothesis 1: Transparency and Interpretability Are Key for ML and AutoML in Many Applications and on Many Levels
  7. Hypothesis 2: Customizability and Flexibility Are Essential to Leverage the Potential of AutoML for Different User Groups
  8. Hypothesis 3: AutoML Tools Have to Integrate with the Data Science Workflow Allowing for an Iterative Interaction with the User
  9. Hypothesis 4: Since Human Experts Are Essential to ML Processes, AutoML Will Only Reach Its Full Potential by Collaborating with Them
  10. Hypothesis 5: Human-Centered AutoML Empowers Users Instead of Making Them Dependent on a System They Do Not Understand
  11. Ongoing Work and Future Directions
  12. Existing Work on Trust and Interpretability in the Context of AutoML
  13. Bridging the Gap Between Algorithmic and HCI AutoML Research
  14. Better Interfaces for AutoML
  15. From Human-Centered AutoML to Human-Centered Automated Data Science
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Selected (one-time) interactions of different user groups with AutoML, ML algorithms and models.
  • Figure 2: CRISP-human-centered AutoML Cycle, inspired by crisp, with a focus on iterative interactions.