Maximizing Reliability with Bayesian Optimization

TL;DR

This work tackles maximizing system reliability under random perturbations by extending Bayesian optimization to rare-event settings. It introduces two acquisition strategies, TS-MR and KG-MR, and develops practical approximations with importance sampling to efficiently estimate extremely small failure probabilities. Empirical results across a suite of low- to high-dimensional problems show that one-shot KG-MR often dominates, with TS-MR providing a strong alternative in many cases. The methods focus sampling near the limit-state surface, improving yield and reliability in expensive, black-box design settings. Overall, the paper advances reliable design optimization by combining BO with rare-event techniques and targeted information gathering.

Abstract

Bayesian optimization (BO) is a popular, sample-efficient technique for expensive, black-box optimization. One such problem arising in manufacturing is that of maximizing the reliability, or equivalently minimizing the probability of a failure, of a design which is subject to random perturbations - a problem that can involve extremely rare failures ($P_\mathrm{fail} = 10^{-6}-10^{-8}$). In this work, we propose two BO methods based on Thompson sampling and knowledge gradient, the latter approximating the one-step Bayes-optimal policy for minimizing the logarithm of the failure probability. Both methods incorporate importance sampling to target extremely small failure probabilities. Empirical results show the proposed methods outperform existing methods in both extreme and non-extreme regimes.

Figures (8)

• Figure 1: Contour plots showing the Styblinski-Tang (2D) problem where the nominal design is perturbed by adding a normally distributed random variable, and perturbed designs with a value above the threshold are classed as failures. The left and middle panels show the black-box function $f$ and the threshold level set, along with the observations collected by one-shot knowledge gradient for maximal reliability (KG-MR) and the algorithm of huang2010egoReliability. The optimal nominal design is shown by the pink dot, surrounded by an ellipse indicating one standard deviation of the normally distributed perturbation. The right panel shows the probability of failure over the domain, as defined in \ref{['eq:fail-prob-additive']}.
• Figure 2: Failure probabilities for the 12 test problems. The first column contains the GP test problems, while the remaining columns are problems with potential for model mismatch. The last column contains problems for which KG-MR is not expected to have an advantage. The failure probability associated with the recommended solution is shown as a function of number of evaluations of the expensive black-box function. The solid lines show the median failure probability over 30 repeats and the shaded regions show the upper and lower quartiles.
• Figure 3: Failure probabilities of the two EGRA variants for the 12 test problems in \ref{['fig:results-rare']}. The failure probability associated with the recommended solution is shown as a function of number of evaluations of the expensive black-box function. The solid lines show the median failure probability over 30 repeats and the shaded regions show the upper and lower quartiles.
• Figure 4: The approximation $1 - \iota({\bm{g}}({\bm{x}}, {\bm{u}}); \delta)$ of $\mathbb{I}_{\{{\bm{g}}({\bm{x}}, {\bm{u}}) \notin {\mathcal{Y}}_\mathrm{feas}\}}$ is shown as a function of ${\bm{g}}({\bm{x}}, {\bm{u}})$. This is defined in \ref{['eq:bounds-indicator-smoothing']} and used to smooth discontinuities in \ref{['eq:pn-naive-approx']}.
• Figure 5: The first 30 sampling locations for five algorithms tested on the 2D GP test problem. The rows show three different runs, corresponding to three different initial designs.

Theorems & Definitions (9)

• Lemma 2.1
• Lemma 5.1
• Remark 6.1
• Remark 6.2
• Lemma 6.2
• proof
• proof
• Lemma 1.1
• proof