Nonmyopic Global Optimisation via Approximate Dynamic Programming
Filippo Airaldi, Bart De Schutter, Azita Dabiri
TL;DR
This work tackles global optimisation of expensive, gradient-free functions by extending nonmyopic acquisition concepts from Bayesian to deterministic surrogate frameworks. It develops dynamic programming-based lookahead strategies— rollout and multi-step scenario-based optimisation—tailored to IDW and RBF surrogates, including explicit surrogate dynamics and a sampling mechanism that yields GP-like posteriors. A carefully crafted reward function and horizon-based acquisitions enable planning over multiple future evaluations, leading to improved convergence over traditional myopic methods on both synthetic benchmarks and real-world hyperparameter tuning, including data-driven MPC tuning for a chemical reactor. The results demonstrate that nonmyopic deterministic GO can outperform Greedy GO with meaningful gains in final optimisation quality, at the cost of higher computational demand, which can be mitigated with parallel hardware and scalable sampling strategies.
Abstract
Unconstrained global optimisation aims to optimise expensive-to-evaluate black-box functions without gradient information. Bayesian optimisation, one of the most well-known techniques, typically employs Gaussian processes as surrogate models, leveraging their probabilistic nature to balance exploration and exploitation. However, Gaussian processes become computationally prohibitive in high-dimensional spaces. Recent alternatives, based on inverse distance weighting (IDW) and radial basis functions (RBFs), offer competitive, computationally lighter solutions. Despite their efficiency, both traditional global and Bayesian optimisation strategies suffer from the myopic nature of their acquisition functions, which focus solely on immediate improvement neglecting future implications of the sequential decision making process. Nonmyopic acquisition functions devised for the Bayesian setting have shown promise in improving long-term performance. Yet, their use in deterministic strategies with IDW and RBF remains unexplored. In this work, we introduce novel nonmyopic acquisition strategies tailored to IDW- and RBF-based global optimisation. Specifically, we develop dynamic programming-based paradigms, including rollout and multi-step scenario-based optimisation schemes, to enable lookahead acquisition. These methods optimise a sequence of query points over a horizon (instead of only at the next step) by predicting the evolution of the surrogate model, inherently managing the exploration-exploitation trade-off in a systematic way via optimisation techniques. The proposed approach represents a significant advance in extending nonmyopic acquisition principles, previously confined to Bayesian optimisation, to the deterministic framework. Empirical results on synthetic and hyperparameter tuning benchmark problems demonstrate that these nonmyopic methods outperform conventional myopic approaches.