Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conservative Surrogate Models for Optimization with the Active Subspace Method

Philippe-André Luneau

TL;DR

High-dimensional optimization with nonlinear constraints is expensive; the paper proposes conservative surrogates built on the Active Subspace Method combined with Gaussian process regression to reduce cost while preserving feasibility; conservativeness is achieved by biasing the surrogate training data via bootstrap or concentration-inequality based procedures to ensure a user-specified probability of overestimation; the approach is demonstrated on a toy thermal-design problem and a two-material diffusion design, showing reduced constraint violations with negligible extra computation; these results indicate a practical pathway to reliable, low-cost optimization in high-dimensional settings.

Abstract

We are interested in building low-dimensional surrogate models to reduce optimization costs, while having theoretical guarantees that the optimum will satisfy the constraints of the full-size model, by making conservative approximations. The surrogate model is constructed using a Gaussian process regression (GPR). To ensure conservativeness, two new approaches are proposed: the first one using bootstrapping, and the second one using concentration inequalities. Those two techniques are based on a stochastic argument and thus will only enforce conservativeness up to a user-defined probability threshold. The method has applications in the context of optimization using the active subspace method for dimensionality reduction of the objective function and the constraints, addressing recorded issues about constraint violations. The resulting algorithms are tested on a toy optimization problem in thermal design.

Conservative Surrogate Models for Optimization with the Active Subspace Method

TL;DR

High-dimensional optimization with nonlinear constraints is expensive; the paper proposes conservative surrogates built on the Active Subspace Method combined with Gaussian process regression to reduce cost while preserving feasibility; conservativeness is achieved by biasing the surrogate training data via bootstrap or concentration-inequality based procedures to ensure a user-specified probability of overestimation; the approach is demonstrated on a toy thermal-design problem and a two-material diffusion design, showing reduced constraint violations with negligible extra computation; these results indicate a practical pathway to reliable, low-cost optimization in high-dimensional settings.

Abstract

We are interested in building low-dimensional surrogate models to reduce optimization costs, while having theoretical guarantees that the optimum will satisfy the constraints of the full-size model, by making conservative approximations. The surrogate model is constructed using a Gaussian process regression (GPR). To ensure conservativeness, two new approaches are proposed: the first one using bootstrapping, and the second one using concentration inequalities. Those two techniques are based on a stochastic argument and thus will only enforce conservativeness up to a user-defined probability threshold. The method has applications in the context of optimization using the active subspace method for dimensionality reduction of the objective function and the constraints, addressing recorded issues about constraint violations. The resulting algorithms are tested on a toy optimization problem in thermal design.
Paper Structure (14 sections, 5 theorems, 43 equations, 10 figures, 2 tables, 2 algorithms)

This paper contains 14 sections, 5 theorems, 43 equations, 10 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Theoretical setting
  3. Active Subspace Method (ASM) with kriging
  4. Active subspace optimization
  5. Methodology
  6. Biased ASM
  7. Probability estimation using Chernoff bounds
  8. Probability estimation using bootstrap
  9. Estimation of $S^{(\beta)}$
  10. Numerical Experiments
  11. Feasible regions analysis
  12. Limitations
  13. Two-Material Optimal Distribution Problem
  14. Conclusion

Key Result

Theorem 2.1

Given that: then $\norm{f - f_\mathrm{GPR}}_{2,\rho}$ is bounded above by where $C_1=C_1(X,\rho)$.

Figures (10)

  • Figure 1: The active subspace $\mathcal{Y}\subset \mathbb{R}$ in $X=[-1,1]^3$, with the inactive subspace $\mathcal{Z}\subset \mathbb{R}^2$. The black dot represents the origin.
  • Figure 2: The active (white) and inactive (black) subspaces of the function $G$.
  • Figure 3: (a) Sampling of $G$ projected over the active subspace. (b) The GPR fitted over the active subspace.
  • Figure 4: Constraints to be enforced when pulling back the solution to the fullspace. The exact constraints are in black, the GPR constraints are in red and the biased constraints (with $\tau = 0.95$) are in blue. The dashed lines indicates the inside of the feasible region.
  • Figure 5: Feasible sets obtained for $\tau = 0.5$ for the different approaches.
  • ...and 5 more figures

Theorems & Definitions (8)

  • Theorem 2.1
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • proof
  • Theorem 3.4: Chernoff Bound