Machine Learning Algorithms for Improving Black Box Optimization Solvers

TL;DR

The paper interrogates how machine learning and reinforcement learning can enhance black-box optimization (BBO) solvers, which operate with costly evaluations and no gradient information. It surveys surrogate-based, zero-order, and ML/RL–driven enhancements, including mlrMBO, ZO-AdaMM, ABBO, SPBOpt, B2Opt, DiffBBO, Surr-RLDE, RBO, CAS-MORE, LB-SGD, PIBB, and Q-Mamba, and it connects these to benchmarking efforts like the NeurIPS 2020 BBO Challenge and MetaBox. The contributions span a taxonomy of classical BBO methods, detailed ML/RL extensions that improve scalability, robustness, and adaptability, and a discussion of standardized benchmarks to compare approaches fairly. The findings suggest that ML/RL do not replace classical solvers but transform them into more scalable, robust, and adaptive frameworks for real-world optimization, with notable gains in sample efficiency, resilience to noise, and multi-task transfer. The paper highlights practical impact for engineering and ML applications where evaluation budgets are tight, and points to future work in mixed-integer domains, improved generalization, and interpretable decision-making under uncertainty.

Abstract

Black-box optimization (BBO) addresses problems where objectives are accessible only through costly queries without gradients or explicit structure. Classical derivative-free methods -- line search, direct search, and model-based solvers such as Bayesian optimization -- form the backbone of BBO, yet often struggle in high-dimensional, noisy, or mixed-integer settings. Recent advances use machine learning (ML) and reinforcement learning (RL) to enhance BBO: ML provides expressive surrogates, adaptive updates, meta-learning portfolios, and generative models, while RL enables dynamic operator configuration, robustness, and meta-optimization across tasks. This paper surveys these developments, covering representative algorithms such as NNs with the modular model-based optimization framework (mlrMBO), zeroth-order adaptive momentum methods (ZO-AdaMM), automated BBO (ABBO), distributed block-wise optimization (DiBB), partition-based Bayesian optimization (SPBOpt), the transformer-based optimizer (B2Opt), diffusion-model-based BBO, surrogate-assisted RL for differential evolution (Surr-RLDE), robust BBO (RBO), coordinate-ascent model-based optimization with relative entropy (CAS-MORE), log-barrier stochastic gradient descent (LB-SGD), policy improvement with black-box (PIBB), and offline Q-learning with Mamba backbones (Q-Mamba). We also review benchmark efforts such as the NeurIPS 2020 BBO Challenge and the MetaBox framework. Overall, we highlight how ML and RL transform classical inexact solvers into more scalable, robust, and adaptive frameworks for real-world optimization.