Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bandit Approach with Evolutionary Operators for Model Selection

Margaux Brégère, Julie Keisler

TL;DR

The paper reframes AutoML model selection as best-arm identification in an infinite-armed bandit with budget $T$, where each arm corresponds to a potential model and the reward is validation accuracy after partial training. It introduces Mutant-UCB, a novel UCB-E–based algorithm augmented with an evolutionary mutation operator that generates new candidate models from promising ones, enabling efficient search without strong assumptions about the search space or reward structure. Mutant-UCB balances exploration via evaluating many models and exploitation via mutations that probe nearby configurations, using a maximum per-model training limit $N$ and a mutation probability $p_t$. Empirical results on three image-classification datasets show Mutant-UCB outperforming Random Search, an asynchronous Evolutionary Algorithm, and Hyperband, with faster convergence and higher final accuracies, demonstrating robust performance in HPC environments and offering broad applicability to diverse search spaces.

Abstract

This work formulates model selection as an infinite-armed bandit problem, namely, a problem in which a decision maker iteratively selects one of an infinite number of fixed choices (i.e., arms) when the properties of each choice are only partially known at the time of allocation and may become better understood over time, via the attainment of rewards.Here, the arms are machine learning models to train and selecting an arm corresponds to a partial training of the model (resource allocation).The reward is the accuracy of the selected model after its partial training.We aim to identify the best model at the end of a finite number of resource allocations and thus consider the best arm identification setup. We propose the algorithm Mutant-UCB that incorporates operators from evolutionary algorithms into the UCB-E (Upper Confidence Bound Exploration) bandit algorithm introduced by Audiber et al.Tests carried out on three open source image classification data sets attest to the relevance of this novel combining approach, which outperforms the state-of-the-art for a fixed budget.

A Bandit Approach with Evolutionary Operators for Model Selection

TL;DR

The paper reframes AutoML model selection as best-arm identification in an infinite-armed bandit with budget , where each arm corresponds to a potential model and the reward is validation accuracy after partial training. It introduces Mutant-UCB, a novel UCB-E–based algorithm augmented with an evolutionary mutation operator that generates new candidate models from promising ones, enabling efficient search without strong assumptions about the search space or reward structure. Mutant-UCB balances exploration via evaluating many models and exploitation via mutations that probe nearby configurations, using a maximum per-model training limit and a mutation probability . Empirical results on three image-classification datasets show Mutant-UCB outperforming Random Search, an asynchronous Evolutionary Algorithm, and Hyperband, with faster convergence and higher final accuracies, demonstrating robust performance in HPC environments and offering broad applicability to diverse search spaces.

Abstract

This work formulates model selection as an infinite-armed bandit problem, namely, a problem in which a decision maker iteratively selects one of an infinite number of fixed choices (i.e., arms) when the properties of each choice are only partially known at the time of allocation and may become better understood over time, via the attainment of rewards.Here, the arms are machine learning models to train and selecting an arm corresponds to a partial training of the model (resource allocation).The reward is the accuracy of the selected model after its partial training.We aim to identify the best model at the end of a finite number of resource allocations and thus consider the best arm identification setup. We propose the algorithm Mutant-UCB that incorporates operators from evolutionary algorithms into the UCB-E (Upper Confidence Bound Exploration) bandit algorithm introduced by Audiber et al.Tests carried out on three open source image classification data sets attest to the relevance of this novel combining approach, which outperforms the state-of-the-art for a fixed budget.
Paper Structure (22 sections, 2 equations, 3 figures, 1 table, 2 algorithms)

This paper contains 22 sections, 2 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Model selection problem setup: a bandit approach
  3. Literature Discussion
  4. Evolutionary methods.
  5. Bayesian optimization.
  6. Bandits approaches.
  7. Best-arm identification in infinite-armed bandits.
  8. Contributions
  9. Set-up
  10. A UCB-based algorithm incorporating mutation operators from evolutionary algorithms
  11. A brief reminder of the UCB-E algorithm
  12. Main contribution: the Mutant-UCB algorithm
  13. Experiments
  14. Experiment design
  15. Data sets.
  16. ...and 7 more sections

Figures (3)

  • Figure 2: Accuracy of the best model over computational time for Random Search (RS), asynchronous evolutionary algorithm (EA) and Mutant-UCB on CIFAR-10, MRBI and SVHN data sets.
  • Figure 3: Best model found for CIFAR-10, MRBI and SVHN by Random Search and Hyperband
  • Figure 7: Best configurations found by the evolutionary algorithm and Mutant-UCB on the CIFAR-10 data set.

Theorems & Definitions (1)

  • Remark 3.1