Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble

Hanyang Wang, Juergen Branke, Matthias Poloczek

TL;DR

This work tackles efficient optimization of expensive multi-objective problems under decision-maker preferences by modeling the DM's utility with a Monotonic Neural Network Ensemble (MoNNE). MoNNE enforces monotonicity via positive-weight transformations, learns from pairwise comparisons with hinge loss, and provides uncertainty through an ensemble, integrated into BOPE with a modified IEUBO acquisition for preference queries and qNEIUU for experimentation. Empirical results show that BOPE-MoNNE consistently outperforms GP-based and PBO baselines, is robust to utility-noise, and benefits from the combination of monotonicity and ensemble uncertainty through ablation studies. This yields a practical, scalable approach for interactive, preference-guided multi-objective optimization with real-world applicability in noisy settings.

Abstract

Many real-world black-box optimization problems have multiple conflicting objectives. Rather than attempting to approximate the entire set of Pareto-optimal solutions, interactive preference learning allows to focus the search on the most relevant subset. However, few previous studies have exploited the fact that utility functions are usually monotonic. In this paper, we address the Bayesian Optimization with Preference Exploration (BOPE) problem and propose using a neural network ensemble as a utility surrogate model. This approach naturally integrates monotonicity and supports pairwise comparison data. Our experiments demonstrate that the proposed method outperforms state-of-the-art approaches and exhibits robustness to noise in utility evaluations. An ablation study highlights the critical role of monotonicity in enhancing performance.

Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble

TL;DR

This work tackles efficient optimization of expensive multi-objective problems under decision-maker preferences by modeling the DM's utility with a Monotonic Neural Network Ensemble (MoNNE). MoNNE enforces monotonicity via positive-weight transformations, learns from pairwise comparisons with hinge loss, and provides uncertainty through an ensemble, integrated into BOPE with a modified IEUBO acquisition for preference queries and qNEIUU for experimentation. Empirical results show that BOPE-MoNNE consistently outperforms GP-based and PBO baselines, is robust to utility-noise, and benefits from the combination of monotonicity and ensemble uncertainty through ablation studies. This yields a practical, scalable approach for interactive, preference-guided multi-objective optimization with real-world applicability in noisy settings.

Abstract

Many real-world black-box optimization problems have multiple conflicting objectives. Rather than attempting to approximate the entire set of Pareto-optimal solutions, interactive preference learning allows to focus the search on the most relevant subset. However, few previous studies have exploited the fact that utility functions are usually monotonic. In this paper, we address the Bayesian Optimization with Preference Exploration (BOPE) problem and propose using a neural network ensemble as a utility surrogate model. This approach naturally integrates monotonicity and supports pairwise comparison data. Our experiments demonstrate that the proposed method outperforms state-of-the-art approaches and exhibits robustness to noise in utility evaluations. An ablation study highlights the critical role of monotonicity in enhancing performance.

Paper Structure

This paper contains 24 sections, 1 theorem, 14 equations, 22 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Problem Setting
  4. Utility Surrogate Model and Pairwise Comparison Acquisition Function
  5. Monotonic Neural Network Ensemble as Utility Surrogate Model
  6. Acquisition Functions
  7. Experimentation
  8. Preference Exploration
  9. Experimental Results
  10. Conclusion
  11. Appendix
  12. Algorithm
  13. 'Vanilla' BO by Neural Network Ensemble and HMC-trained BNN
  14. HMC-trained BNN Given Pairwise Comparisons
  15. Model Visualization Given Different Numbers of Pairwise Comparisons
  16. ...and 9 more sections

Key Result

Theorem 1

(sill1997monotonic) Let $m(x)$ be any continuous, bounded monotonic function with bounded partial derivatives, mapping $[0,1]^d$ to $\mathbb{R}$. Then there exists a function $m_{net}(x)$ which can be implemented by a monotonic network and is such that, for any $\epsilon$ and any $x \in[0,1]^d,\left

Figures (22)

  • Figure 1: The experiment alternates between the Preference Exploration and the Experimentation stages.
  • Figure 2: In each row, the left plot depicts the true utility function slices; the middle is the GP surrogate model and the right one is the MoNNE model. While the scale varies significantly across models, this does not affect our analysis, as we care about relative rankings rather than absolute values.
  • Figure 3: Experimental results show that among all BOPE algorithms, BOPE-MoNNE demonstrates superior performance, followed by BOPE-BMNN and BOPE-GP.
  • Figure 4: The ablation study shows that MoNNE performs best, highlighting the importance of all components. Regret values capped at $10^{-5}$.
  • Figure 5: HMC-trained BNNs share similar performance as neural network ensemble in 'vanilla BO' setting.
  • ...and 17 more figures

Theorems & Definitions (1)

  • Theorem 1