Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data-Driven Bayesian Nonparametric Approach for Black-Box Optimization

Haowei Wang, Xun Zhang, Szu Hui Ng, Songhao Wang

TL;DR

It is shown that the DaBNO objective formulation can converge to the true objective asymptotically and a surrogate-assisted algorithm DaBNO-K is developed to efficiently optimize the proposed objective function based on a carefully designed kernel.

Abstract

We present a data-driven Bayesian nonparametric approach for global optimization (DaBNO) of stochastic black-box function. The function value depends on the distribution of a random vector. However, this distribution is usually complex and hardly known in practice, and is often inferred from data (realizations of random vectors). The DaBNO accounts for the finite-data error that arises when estimating the distribution and relaxes the commonly-used parametric assumption to reduce the distribution-misspecified error. We show that the DaBNO objective formulation can converge to the true objective asymptotically. We further develop a surrogate-assisted algorithm DaBNO-K to efficiently optimize the proposed objective function based on a carefully designed kernel. Numerical experiments are conducted with several synthetic and practical problems, demonstrating the empirical global convergence of this algorithm and its finite-sample performance.

A Data-Driven Bayesian Nonparametric Approach for Black-Box Optimization

TL;DR

It is shown that the DaBNO objective formulation can converge to the true objective asymptotically and a surrogate-assisted algorithm DaBNO-K is developed to efficiently optimize the proposed objective function based on a carefully designed kernel.

Abstract

We present a data-driven Bayesian nonparametric approach for global optimization (DaBNO) of stochastic black-box function. The function value depends on the distribution of a random vector. However, this distribution is usually complex and hardly known in practice, and is often inferred from data (realizations of random vectors). The DaBNO accounts for the finite-data error that arises when estimating the distribution and relaxes the commonly-used parametric assumption to reduce the distribution-misspecified error. We show that the DaBNO objective formulation can converge to the true objective asymptotically. We further develop a surrogate-assisted algorithm DaBNO-K to efficiently optimize the proposed objective function based on a carefully designed kernel. Numerical experiments are conducted with several synthetic and practical problems, demonstrating the empirical global convergence of this algorithm and its finite-sample performance.

Paper Structure

This paper contains 33 sections, 6 theorems, 83 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. Literature Review
  4. Our Contributions
  5. Objective function for DaBNO and its asymptotics
  6. The DaBNO objective function
  7. Asymptotic properties of the DaBNO objective function
  8. Kriging surrogate model for DaBNO
  9. Stochastic Kriging model of $f(\mathbf{x},\mathbf{P})$
  10. Approximate model for $g(\mathbf{x})$
  11. DaBNO-K algorithm
  12. Convergence analysis of the DaBNO-K algorithm
  13. Experiments
  14. Empirical convergence results
  15. Setup of the inventory problem
  16. ...and 18 more sections

Key Result

Lemma 1

Suppose Assumption assum holds. Then

Figures (7)

  • Figure 1: Empirical convergence of the optimal value/solution of the DaBNO objective to the true optimal value/solution as $s$ increases
  • Figure 2: Empirical performance in terms of gGAP and gxGAP of our surrogate based algorithm as the size of the budget increases. The number of real-world data $s$ is $1000$.
  • Figure 3: Empirical performance in terms of GAP and xGAP of our surrogate based algorithm as the size of the budget increases. The number of real-world data $s$ is $1000$.
  • Figure 4: The Griewank and StybTang function : the boxplot of the GAP values for three approaches (DaBNO-K, hist, parametric-exp and parametric-lognorm) under different uncertainty levels of the distribution of $\BFu$
  • Figure 5: Critical care facility
  • ...and 2 more figures

Theorems & Definitions (8)

  • Lemma 1
  • Theorem 1
  • Theorem 2
  • Remark 1
  • Theorem 3
  • Corollary 1
  • Corollary 2
  • Remark 2