NUBO: A Transparent Python Package for Bayesian Optimization

TL;DR

NUBO addresses the need for accessible Bayesian optimization of expensive black-box functions by delivering a transparent, modular Python package built on Torch and GPyTorch. It combines GP surrogates with analytical and Monte Carlo acquisition functions to support sequential, parallel, asynchronous, constrained, and mixed optimization in bounded spaces. The paper demonstrates competitive performance against more complex BO tools while emphasizing simplicity, readability, and reproducibility, illustrated through a detailed six-dimensional case study. Practically, NUBO lowers the barrier to applying Bayesian optimization in scientific computing and experiments, enabling researchers to tailor optimization loops with building blocks and clear documentation. The work suggests future directions toward multi-fidelity, multi-objective, and high-dimensional scenarios to broaden applicability.

Abstract

NUBO, short for Newcastle University Bayesian Optimization, is a Bayesian optimization framework for optimizing expensive-to-evaluate black-box functions, such as physical experiments and computer simulators. Bayesian optimization is a cost-efficient optimization strategy that uses surrogate modeling via Gaussian processes to represent an objective function and acquisition functions to guide the selection of candidate points to approximate the global optimum of the objective function. NUBO focuses on transparency and user experience to make Bayesian optimization accessible to researchers from all disciplines. Clean and understandable code, precise references, and thorough documentation ensure transparency, while a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimization algorithms ensure a good user experience. NUBO allows users to tailor Bayesian optimization to their problem by writing a custom optimization loop using the provided building blocks. It supports sequential single-point, parallel multi-point, and asynchronous optimization of bounded, constrained, and mixed (discrete and continuous) parameter input spaces. Only algorithms and methods extensively tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessarily large number of options. The package is written in Python but does not require expert knowledge of Python to optimize simulators and experiments. NUBO is distributed as open-source software under the BSD 3-Clause license.

Figures (5)

• Figure 1: Comparison of different Python packages for Bayesian optimization. A) Sequential single-point optimization on the 2D Levy function; B) Sequential single-point optimization on the 6D Hartmann function; C) Parallel multi-point optimization with a batch size of four on the 2D Levy function; D) Parallel multi-point optimization with a batch size of four on the 6D Hartmann function.
• Figure 2: Bayesian optimization applied to a 1-dimensional function with one local and one global maximum. Upper confidence bound is used as the acquisition function. The input space is bounded by $[0, 10]$.
• Figure 3: NUBO flowchart. Overview of the recommended algorithms for specific problems. Start in yellow, decisions in blue, and recommended algorithm in green.
• Figure 4: Latin hypercube sampling compared to random sampling.
• Figure 5: Results of the Bayesian optimization algorithm implemented with NUBO, as defined in this case study, compared to random sampling and Latin hypercube sampling.