Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolving machine learning workflows through interactive AutoML

Rafael Barbudo, Aurora Ramírez, José Raúl Romero

TL;DR

This work tackles automatic workflow composition (AWC) in AutoML by introducing iEvoFlow, an interactive grammar-guided genetic programming approach that lets users prune the underlying context-free grammar to steer the search toward regions aligned with user preferences. By combining CFG-based GP with human-in-the-loop optimisation, iEvoFlow preserves high predictive performance while reducing evaluation time, as demonstrated by simulated-user studies and a real-user experiment with 20 participants. Key findings show that human intervention can improve either accuracy or speed for many participants, with several achieving substantial speedups and others discovering novel, efficient pipelines not found by the non-interactive baseline. The results highlight the practical value of human-guided AutoML for democratizing access to powerful workflow discovery and customization, and point to future work on expanding interactive capabilities and guidance.

Abstract

Automatic workflow composition (AWC) is a relevant problem in automated machine learning (AutoML) that allows finding suitable sequences of preprocessing and prediction models together with their optimal hyperparameters. This problem can be solved using evolutionary algorithms and, in particular, grammar-guided genetic programming (G3P). Current G3P approaches to AWC define a fixed grammar that formally specifies how workflow elements can be combined and which algorithms can be included. In this paper we present \ourmethod, an interactive G3P algorithm that allows users to dynamically modify the grammar to prune the search space and focus on their regions of interest. Our proposal is the first to combine the advantages of a G3P method with ideas from interactive optimisation and human-guided machine learning, an area little explored in the context of AutoML. To evaluate our approach, we present an experimental study in which 20 participants interact with \ourmethod to evolve workflows according to their preferences. Our results confirm that the collaboration between \ourmethod and humans allows us to find high-performance workflows in terms of accuracy that require less tuning time than those found without human intervention.

Evolving machine learning workflows through interactive AutoML

TL;DR

This work tackles automatic workflow composition (AWC) in AutoML by introducing iEvoFlow, an interactive grammar-guided genetic programming approach that lets users prune the underlying context-free grammar to steer the search toward regions aligned with user preferences. By combining CFG-based GP with human-in-the-loop optimisation, iEvoFlow preserves high predictive performance while reducing evaluation time, as demonstrated by simulated-user studies and a real-user experiment with 20 participants. Key findings show that human intervention can improve either accuracy or speed for many participants, with several achieving substantial speedups and others discovering novel, efficient pipelines not found by the non-interactive baseline. The results highlight the practical value of human-guided AutoML for democratizing access to powerful workflow discovery and customization, and point to future work on expanding interactive capabilities and guidance.

Abstract

Automatic workflow composition (AWC) is a relevant problem in automated machine learning (AutoML) that allows finding suitable sequences of preprocessing and prediction models together with their optimal hyperparameters. This problem can be solved using evolutionary algorithms and, in particular, grammar-guided genetic programming (G3P). Current G3P approaches to AWC define a fixed grammar that formally specifies how workflow elements can be combined and which algorithms can be included. In this paper we present \ourmethod, an interactive G3P algorithm that allows users to dynamically modify the grammar to prune the search space and focus on their regions of interest. Our proposal is the first to combine the advantages of a G3P method with ideas from interactive optimisation and human-guided machine learning, an area little explored in the context of AutoML. To evaluate our approach, we present an experimental study in which 20 participants interact with \ourmethod to evolve workflows according to their preferences. Our results confirm that the collaboration between \ourmethod and humans allows us to find high-performance workflows in terms of accuracy that require less tuning time than those found without human intervention.
Paper Structure (33 sections, 5 figures, 3 tables)

This paper contains 33 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Evolutionary workflow composition
  4. Interactive machine learning
  5. Related work
  6. Description of iEvoFlow
  7. The evolutionary search process
  8. Interactive approach
  9. Selection of solutions
  10. Type of user feedback
  11. Interaction schema
  12. Preference information lifetime
  13. Interactive interface
  14. Experimental methodology
  15. Research questions
  16. ...and 18 more sections

Figures (5)

  • Figure 1: Proposed interactive evolutionary model.
  • Figure 2: Graphical user interface of iEvoFlow.
  • Figure 3: Speedup achieved with simulated user interactions.
  • Figure 4: Number of times each algorithm or hyperparameter value has been deleted by participants in each interaction.
  • Figure 5: Responses of the participants to the questionnaire.