Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivative-Free Bound-Constrained Optimization for Solving Structured Problems with Surrogate Models

Frank E. Curtis, Shima Dezfulian, Andreas Wächter

TL;DR

This work proposes and analyzes a model-based derivative-free DFO algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions and proposes and analyze a new algorithm for obtaining interpolation points that handles unrelaxable bound constraints.

Abstract

We propose and analyze a model-based derivative-free (DFO) algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions. We assume that the black-box functions are smooth and the evaluation of them is the computational bottleneck of the algorithm. The distinguishing feature of our algorithm is the use of approximate function values at interpolation points which can be obtained by an application-specific surrogate model that is cheap to evaluate. As an example, we consider the situation in which a sequence of related optimization problems is solved and present a regression-based approximation scheme that uses function values that were evaluated when solving prior problem instances. In addition, we propose and analyze a new algorithm for obtaining interpolation points that handles unrelaxable bound constraints. Our numerical results show that our algorithm outperforms a state-of-the-art DFO algorithm for solving a least-squares problem from a chemical engineering application when a history of black-box function evaluations is available.

Derivative-Free Bound-Constrained Optimization for Solving Structured Problems with Surrogate Models

TL;DR

This work proposes and analyzes a model-based derivative-free DFO algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions and proposes and analyze a new algorithm for obtaining interpolation points that handles unrelaxable bound constraints.

Abstract

We propose and analyze a model-based derivative-free (DFO) algorithm for solving bound-constrained optimization problems where the objective function is the composition of a smooth function and a vector of black-box functions. We assume that the black-box functions are smooth and the evaluation of them is the computational bottleneck of the algorithm. The distinguishing feature of our algorithm is the use of approximate function values at interpolation points which can be obtained by an application-specific surrogate model that is cheap to evaluate. As an example, we consider the situation in which a sequence of related optimization problems is solved and present a regression-based approximation scheme that uses function values that were evaluated when solving prior problem instances. In addition, we propose and analyze a new algorithm for obtaining interpolation points that handles unrelaxable bound constraints. Our numerical results show that our algorithm outperforms a state-of-the-art DFO algorithm for solving a least-squares problem from a chemical engineering application when a history of black-box function evaluations is available.
Paper Structure (23 sections, 21 theorems, 63 equations, 1 figure, 4 algorithms)

This paper contains 23 sections, 21 theorems, 63 equations, 1 figure, 4 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Notation
  4. Problem Formulation
  5. Multi-Output Simulation-based Optimization with Surrogate Models
  6. Sequence of Optimization Problems
  7. Literature Review
  8. Organization
  9. A Derivative-Free Algorithmic Framework
  10. Fully Linear Models
  11. Algorithm Description
  12. Convergence Result
  13. Interpolation Set Construction
  14. Generating Interpolation Points for Bound Constraints
  15. Function Approximation Using Regression
  16. ...and 8 more sections

Key Result

Lemma 2.4

Suppose that Assumptions assumption:F_h and assumption:q hold and consider arbitrary $k \in \mathbb{N}$ and ${\@fontswitch{}{\mathcal{}} D}_k := \{d_1, \dots, d_{n_x}\} \subset B_{\text{\rm tr}}(0, \Delta_k) \cap (\Omega - x_k)$ such that $(a)$ for all $i \in [p]$, the model $q_{k,i}$ satisfies eq:g Then, for any $x \in B_{\text{\rm tr}}(x_k; \Delta_k) \cap \Omega$ and any $i \in [p]$, the model $

Figures (1)

  • Figure 1: $\text{\rm Alg}_{\mathcal{H}}$ and $\text{\rm Alg}_{\emptyset}$ comparison.

Theorems & Definitions (24)

  • Definition 2.3
  • Lemma 2.4
  • Lemma 2.5
  • Lemma 2.6
  • Remark 1
  • Theorem 2.8
  • Remark 2
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • ...and 14 more