Put CASH on Bandits: A Max K-Armed Problem for Automated Machine Learning

TL;DR

This work addresses Combined Algorithm Selection and Hyperparameter Optimization (CASH) by recasting it as a two-level bandit problem aimed at maximizing the maximum observed reward rather than the average. It introduces MaxUCB, a distribution-adapted extreme-bandit algorithm tailored to bounded, left-skewed reward distributions typical in AutoML, with a theoretically grounded regret bound that relies on the suboptimality gap and distribution-shape constants L and U. The authors complement theory with a data-driven analysis of HPO reward dynamics and demonstrate state-of-the-art performance on four AutoML benchmarks, showing robust early and final results and insightful sensitivity analyses for the exploration parameter α. The approach offers practical gains for AutoML systems by enabling efficient, scalable allocation of HPO budget across model classes and by potentially extending to NAS and sub-supernet selection. Collectively, the work advances extreme-bandit methods for CASH and provides a solid foundation for further integration of bandit-based resource allocation in automated ML pipelines.

Abstract

The Combined Algorithm Selection and Hyperparameter optimization (CASH) is a challenging resource allocation problem in the field of AutoML. We propose MaxUCB, a max k-armed bandit method to trade off exploring different model classes and conducting hyperparameter optimization. MaxUCB is specifically designed for the light-tailed and bounded reward distributions arising in this setting and, thus, provides an efficient alternative compared to classic max k-armed bandit methods assuming heavy-tailed reward distributions. We theoretically and empirically evaluate our method on four standard AutoML benchmarks, demonstrating superior performance over prior approaches. We make our code and data available at https://github.com/amirbalef/CASH_with_Bandits

Figures (40)

• Figure 1: (Left) MaxUCB (blue line) outperformscombined search (black line) to identify the best-performing model class (brown line). (Middle) The irregular distribution of the empirical performance of model classes is left-skewed, and a higher mean may not correspond to a higher maximum. (Right) MaxUCB selects for which model to run one iteration of HPO during two-level optimization.
• Figure 2: (Left) The average empirical survival function of the reward distribution per arm ranked per dataset. Thin lines correspond to segments of the reward sequence and show the distribution change over time. (Right) The average empirical survival function per dataset for the best and worst arm. Thin lines correspond to individual datasets.
• Figure 3: (Left, Middle) L and U for the average survival functions in Figure \ref{['fig:HPO_ecdf_arms_distributions']}. (Right) We highlight the difference to right-skewed Log-normal and Pareto distributions.
• Figure 4: Histogram of $L$ and $U$ values across individual datasets.
• Figure 5: The sensitivity of MaxUCB to hyperparameter $\alpha$, lower is better.

Theorems & Definitions (11)

• Definition 3.1
• Lemma 3.3
• Proposition 4.1
• Theorem 4.2
• Corollary 4.3
• Lemma A.1
• proof
• proof
• proof
• proof