Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AutoML in The Wild: Obstacles, Workarounds, and Expectations

Yuan Sun, Qiurong Song, Xinning Gui, Fenglong Ma, Ting Wang

TL;DR

The paper investigates how AutoML is used in real-world settings and identifies three core obstacles—customizability, transparency, and privacy—that hinder adoption. Through 19 semi-structured interviews, it reveals a range of user-created workarounds, including contextual data inputs, domain-knowledge integration, internal AutoML tools, manual outcome validation, process tracking, and customized visualizations, as well as privacy-preserving practices and legal safeguards. The study also shows that users apply AutoML selectively and situationally, balancing performance expectations, task requirements, and context while partially relying on external resources and collaboration. These findings yield design implications for domain-specific customization, multifaceted transparency, privacy-by-design measures, and stronger cross-disciplinary collaboration to support sustainable human-AutoML co-adaptation in practice.

Abstract

Automated machine learning (AutoML) is envisioned to make ML techniques accessible to ordinary users. Recent work has investigated the role of humans in enhancing AutoML functionality throughout a standard ML workflow. However, it is also critical to understand how users adopt existing AutoML solutions in complex, real-world settings from a holistic perspective. To fill this gap, this study conducted semi-structured interviews of AutoML users (N=19) focusing on understanding (1) the limitations of AutoML encountered by users in their real-world practices, (2) the strategies users adopt to cope with such limitations, and (3) how the limitations and workarounds impact their use of AutoML. Our findings reveal that users actively exercise user agency to overcome three major challenges arising from customizability, transparency, and privacy. Furthermore, users make cautious decisions about whether and how to apply AutoML on a case-by-case basis. Finally, we derive design implications for developing future AutoML solutions.

AutoML in The Wild: Obstacles, Workarounds, and Expectations

TL;DR

The paper investigates how AutoML is used in real-world settings and identifies three core obstacles—customizability, transparency, and privacy—that hinder adoption. Through 19 semi-structured interviews, it reveals a range of user-created workarounds, including contextual data inputs, domain-knowledge integration, internal AutoML tools, manual outcome validation, process tracking, and customized visualizations, as well as privacy-preserving practices and legal safeguards. The study also shows that users apply AutoML selectively and situationally, balancing performance expectations, task requirements, and context while partially relying on external resources and collaboration. These findings yield design implications for domain-specific customization, multifaceted transparency, privacy-by-design measures, and stronger cross-disciplinary collaboration to support sustainable human-AutoML co-adaptation in practice.

Abstract

Automated machine learning (AutoML) is envisioned to make ML techniques accessible to ordinary users. Recent work has investigated the role of humans in enhancing AutoML functionality throughout a standard ML workflow. However, it is also critical to understand how users adopt existing AutoML solutions in complex, real-world settings from a holistic perspective. To fill this gap, this study conducted semi-structured interviews of AutoML users (N=19) focusing on understanding (1) the limitations of AutoML encountered by users in their real-world practices, (2) the strategies users adopt to cope with such limitations, and (3) how the limitations and workarounds impact their use of AutoML. Our findings reveal that users actively exercise user agency to overcome three major challenges arising from customizability, transparency, and privacy. Furthermore, users make cautious decisions about whether and how to apply AutoML on a case-by-case basis. Finally, we derive design implications for developing future AutoML solutions.
Paper Structure (46 sections, 1 figure, 2 tables)

This paper contains 46 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. HCI Research on AutoML
  4. Users' Perceptions of AutoML
  5. AutoML's Transparency
  6. Human-in-The-Loop AutoML
  7. User Agency and AutoML
  8. Human Agency and Automation
  9. Supporting User Agency in ML
  10. Summary
  11. Method
  12. Recruitment and Interviews
  13. Data Analysis
  14. Findings
  15. Acknowledging and Working around AutoML's Limitations
  16. ...and 31 more sections

Figures (1)

  • Figure 1: Obstacles and Workarounds of Using AutoML in Complex, Real-world Settings.