MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions

TL;DR

MOBO-OSD tackles batch multi-objective Bayesian optimization by building an approximated CHIM and generating a diverse set of orthogonal search directions to locate intersection points with the Pareto boundary. It couples MOBO subproblems along each direction with Pareto Front Estimation to densify the Pareto set, and uses Hypervolume Improvement with Kriging Believer to select batches while preserving exploration diversity. Empirical results across 2–6 objectives on synthetic and real-world benchmarks show consistent gains over state-of-the-art baselines in both sequential and batch modes, including improved hypervolume performance and scalable computation. The approach offers a practical, scalable framework for obtaining well-distributed Pareto fronts with limited evaluations, making it attractive for engineering and scientific optimization tasks that require efficient exploration of trade-offs.

Abstract

Bayesian Optimization (BO) is a powerful tool for optimizing expensive black-box objective functions. While extensive research has been conducted on the single-objective optimization problem, the multi-objective optimization problem remains challenging. In this paper, we propose MOBO-OSD, a multi-objective Bayesian Optimization algorithm designed to generate a diverse set of Pareto optimal solutions by solving multiple constrained optimization problems, referred to as MOBO-OSD subproblems, along orthogonal search directions (OSDs) defined with respect to an approximated convex hull of individual objective minima. By employing a well-distributed set of OSDs, MOBO-OSD ensures broad coverage of the objective space, enhancing both solution diversity and hypervolume performance. To further improve the density of the set of Pareto optimal candidate solutions without requiring an excessive number of subproblems, we leverage a Pareto Front Estimation technique to generate additional solutions in the neighborhood of existing solutions. Additionally, MOBO-OSD supports batch optimization, enabling parallel function evaluations to accelerate the optimization process when resources are available. Through extensive experiments and analysis on a variety of synthetic and real-world benchmark functions with two to six objectives, we demonstrate that MOBO-OSD consistently outperforms the state-of-the-art algorithms. Our code implementation can be found at https://github.com/LamNgo1/mobo-osd.

Figures (23)

• Figure 1: Summary of the MOBO-OSD algorithm: (a) The boundary points are defined via the nadir and ideal points; (b) The approximated CHIM is defined from the boundary points and the OSDs are defined on each $\mathcal{U}(\boldsymbol{\beta})$ point and orthogonal to the approximated CHIM; (c) The MOBO-OSD subproblem (Eq. (\ref{['eq:proposed_subproblem']})) is optimized for $\mathbf{x}^\text{OSD}$ candidates; (d) Additional candidates $\mathbf{x}^\text{PFE}$ are generated by locally exploring around $\mathbf{x}^\text{OSD}$. Finally, $\mathbf{x}_\text{next}$ is chosen based on Eq. (\ref{['eq:batch_selection']}).
• Figure 2: Comparison of MOBO-OSD against the SOTA baselines on 5 synthetic and 4 real-world benchmark problems in sequential setting (batch size 1). Note that DGEMO does support problems with $M>3$ objectives. Overall, MOBO-OSD outperforms the baselines.
• Figure 3: Comparison of MOBO-OSD against the SOTA baselines on 5 synthetic and 4 real-world benchmark problems in batch setting (batch size 4). Overall, MOBO-OSD outperform the baselines.
• Figure 4: Ablation study on the effect of parameter $n_\beta$. Full results can be found in the Appendix \ref{['sec-appendix:additional_abl_results']}. Overall, MOBO-OSD is robust against $n_\beta$.
• Figure 5: Comparison of MOBO-OSD against the SOTA baselines on 5 synthetic and 4 real-world benchmark problems in batch setting (batch size 8). Note that qEVHI has limited iterations due to the prohibitively high computational cost and memory required on problems with $M\ge4$ objectives.