Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Certified Multi-Fidelity Zeroth-Order Optimization

Étienne de Montbrun, Sébastien Gerchinovitz

TL;DR

This paper proposes a certified variant of the MFDOO algorithm and derive a bound on its cost complexity for any Lipschitz function, and proves an $f$-dependent lower bound showing that this algorithm has a near-optimal cost complexity.

Abstract

We consider the problem of multi-fidelity zeroth-order optimization, where one can evaluate a function $f$ at various approximation levels (of varying costs), and the goal is to optimize $f$ with the cheapest evaluations possible. In this paper, we study certified algorithms, which are additionally required to output a data-driven upper bound on the optimization error. We first formalize the problem in terms of a min-max game between an algorithm and an evaluation environment. We then propose a certified variant of the MFDOO algorithm and derive a bound on its cost complexity for any Lipschitz function $f$. We also prove an $f$-dependent lower bound showing that this algorithm has a near-optimal cost complexity. As a direct example, we close the paper by addressing the special case of noisy (stochastic) evaluations, which corresponds to $\eps$-best arm identification in Lipschitz bandits with continuously many arms.

Certified Multi-Fidelity Zeroth-Order Optimization

TL;DR

This paper proposes a certified variant of the MFDOO algorithm and derive a bound on its cost complexity for any Lipschitz function, and proves an -dependent lower bound showing that this algorithm has a near-optimal cost complexity.

Abstract

We consider the problem of multi-fidelity zeroth-order optimization, where one can evaluate a function at various approximation levels (of varying costs), and the goal is to optimize with the cheapest evaluations possible. In this paper, we study certified algorithms, which are additionally required to output a data-driven upper bound on the optimization error. We first formalize the problem in terms of a min-max game between an algorithm and an evaluation environment. We then propose a certified variant of the MFDOO algorithm and derive a bound on its cost complexity for any Lipschitz function . We also prove an -dependent lower bound showing that this algorithm has a near-optimal cost complexity. As a direct example, we close the paper by addressing the special case of noisy (stochastic) evaluations, which corresponds to -best arm identification in Lipschitz bandits with continuously many arms.
Paper Structure (31 sections, 12 theorems, 57 equations, 3 algorithms)

This paper contains 31 sections, 12 theorems, 57 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Certified optimization
  3. Setting
  4. Definitions: environments and certified algorithms
  5. Optimization goal
  6. Main contributions and outline of the paper
  7. Related works
  8. Multi-fidelity optimization
  9. Error certificates
  10. Adaptivity to smoothness versus error certificates
  11. Notation
  12. Standard notation
  13. Lipschitz functions, $\varepsilon$-optimal points, layers
  14. Packing number
  15. The complexity quantity $S_{\beta, L}(f, \varepsilon)$
  16. ...and 16 more sections

Key Result

Lemma 2.3

\newlabellemma:certification0 Suppose Assumption assum:diameter holds, and that $f:\mathcal{X} \to \mathbb{R}$ is an $L$-Lipschitz function, with a maximizer $x^\star\in \mathcal{X}$. Then, for any environment $E \in \mathcal{E}(f)$ and any $t \in \mathbb{N}^*$, the quantity $\xi_t$ defined at Lin

Theorems & Definitions (24)

  • Lemma 2.3
  • Proof 1
  • Theorem 2.4
  • Proof 2: Proof of Theorem \ref{['thm:upper_bound']}
  • Theorem 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Proof 3: Proof of Theorem \ref{['thm:lower_bound']}
  • Proposition 4.1
  • ...and 14 more