Trust Region-Based Bayesian Optimisation to Discover Diverse Solutions

TL;DR

This paper addresses the need for diverse, high-quality solutions in expensive black-box optimisation by adapting trust-region Bayesian optimisation (BO) to diversity objectives. It introduces divTuRBO1, which extends TuRBO1 to maintain a minimum distance $\tau$ from a reference diverse set $X_{div}$, and proposes two strategies (divTuRBO1-seq and divTuRBO1-int) to produce $m$ diverse solutions by combining multiple runs. Compared to ROBOT on 24 BBOB functions up to 20 dimensions, the proposed methods generally yield superior diversity-quality trade-offs, particularly in higher dimensions, with budgets and phase settings affecting performance. The results support using divTuRBO1 with a small number of phases (especially the interleaving approach) for scalable, diversity-driven optimisation in practice and highlight potential applications in domains where diverse high-quality solutions are valuable.

Abstract

Bayesian optimisation (BO) is a surrogate-based optimisation technique that efficiently solves expensive black-box functions with small evaluation budgets. Recent studies consider trust regions to improve the scalability of BO approaches when the problem space scales to more dimensions. Motivated by this research, we explore the effectiveness of trust region-based BO algorithms for diversity optimisation in different dimensional black box problems. We propose diversity optimisation approaches extending TuRBO1, which is the first BO method that uses a trust region-based approach for scalability. We extend TuRBO1 as divTuRBO1, which finds an optimal solution while maintaining a given distance threshold relative to a reference solution set. We propose two approaches to find diverse solutions for black-box functions by combining divTuRBO1 runs in a sequential and an interleaving fashion. We conduct experimental investigations on the proposed algorithms and compare their performance with that of the baseline method, ROBOT (rank-ordered Bayesian optimisation with trust regions). We evaluate proposed algorithms on benchmark functions with dimensions 2 to 20. Experimental investigations demonstrate that the proposed methods perform well, particularly in larger dimensions, even with a limited evaluation budget.

Figures (6)

• Figure 1: Results from one run of each algorithm on the first 6 BBOB functions in the 2-D problem space for distance threshold values $\tau=0.1,1.0$ and $2.0$. The red and black crosses show the optimum of the functions and the outcome of the algorithms, respectively.
• Figure 2: Results from one run of each algorithm on the 18 BBOB functions (F7-F24) for the 2-D problem space with distance threshold $\tau=1.0$. The red and black crosses show the optimum of the functions and the outcome of the algorithms, respectively.
• Figure 3: Results for BBOB functions using divTuRBO-seq, divTuRBO-int and ROBOT with different evaluation budgets.
• Figure 4: Results from divTuRBO-int when 1, 5, 10 and 100 phases for benchmark functions under $\tau=0.1$ and $1.0$. The dashed lines provide references to ROBOT results in each setting.
• Figure 5: Results for 10-D functions using divTuRBO-seq, divTuRBO-int and ROBOT with different evaluation budgets.