Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori Certification

Masahito Hayashi

TL;DR

This analysis provides a unified evaluation template for last-iterate function values under a relative-smoothness-type inequality, with an emphasis on trajectory-wise (a posteriori) certification: whenever a verifiable inequality holds along the realized iterates, the authors obtain explicit last-iterate guarantees.

Abstract

We develop a deterministic zeroth-order mirror descent framework by replacing gradients with a general vector field, yielding a vector-field-driven mirror update that preserves Bregman geometry while accommodating derivative-free oracles. Our analysis provides a unified evaluation template for last-iterate function values under a relative-smoothness-type inequality, with an emphasis on trajectory-wise (a posteriori) certification: whenever a verifiable inequality holds along the realized iterates, we obtain explicit last-iterate guarantees. The framework subsumes a broad class of information-geometric algorithms, including generalized Blahut-Arimoto-type updates, by expressing their dynamics through suitable choices of the vector field. We then instantiate the theory with deterministic central finite differences in moderate dimension, where constructing the vector field via deterministic central finite differences requires 2d off-center function values (and one reusable center value), i.e., 2d+1 evaluations in total, where d is the number of input real numbers. In this deterministic finite-difference setting, the key interface property is not classical convexity alone but a punctured-neighborhood generalized star-convexity condition that isolates an explicit resolution-dependent error floor. Establishing this property for the finite-difference vector field reduces to a robust conic dominance design problem; we give an explicit scaling rule ensuring the required uniform dominance on a circular cone. Together, these results expose a hidden geometric structure linking Bregman telescoping identities, deterministic certification, and robust conic geometry in zeroth-order mirror descent.

Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori Certification

TL;DR

This analysis provides a unified evaluation template for last-iterate function values under a relative-smoothness-type inequality, with an emphasis on trajectory-wise (a posteriori) certification: whenever a verifiable inequality holds along the realized iterates, the authors obtain explicit last-iterate guarantees.

Abstract

We develop a deterministic zeroth-order mirror descent framework by replacing gradients with a general vector field, yielding a vector-field-driven mirror update that preserves Bregman geometry while accommodating derivative-free oracles. Our analysis provides a unified evaluation template for last-iterate function values under a relative-smoothness-type inequality, with an emphasis on trajectory-wise (a posteriori) certification: whenever a verifiable inequality holds along the realized iterates, we obtain explicit last-iterate guarantees. The framework subsumes a broad class of information-geometric algorithms, including generalized Blahut-Arimoto-type updates, by expressing their dynamics through suitable choices of the vector field. We then instantiate the theory with deterministic central finite differences in moderate dimension, where constructing the vector field via deterministic central finite differences requires 2d off-center function values (and one reusable center value), i.e., 2d+1 evaluations in total, where d is the number of input real numbers. In this deterministic finite-difference setting, the key interface property is not classical convexity alone but a punctured-neighborhood generalized star-convexity condition that isolates an explicit resolution-dependent error floor. Establishing this property for the finite-difference vector field reduces to a robust conic dominance design problem; we give an explicit scaling rule ensuring the required uniform dominance on a circular cone. Together, these results expose a hidden geometric structure linking Bregman telescoping identities, deterministic certification, and robust conic geometry in zeroth-order mirror descent.
Paper Structure (38 sections, 22 theorems, 121 equations, 2 tables)

This paper contains 38 sections, 22 theorems, 121 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Motivation: deterministic zeroth-order mirror descent for matrix-function objectives
  3. Related work organized by oracle model and geometry
  4. (D1) Deterministic derivative-free optimization: direct search / pattern search
  5. (D2) Deterministic derivative-free optimization: model-based / trust-region methods
  6. (R1) Randomized zeroth-order methods and optimal-rate viewpoints
  7. (G1) Geometry-aware first-order methods: Bregman geometry, relative smoothness, and certification
  8. (Ours) Positioning: deterministic zeroth-order mirror descent via vector fields and trajectory-wise certification
  9. Organization
  10. Generalized Mirror Descent
  11. Relation to generalized Arimoto-Blahut algorithm
  12. A norm-pair toolbox: strong convexity and smoothness
  13. Feasibility of trajectory-wise relative smoothness
  14. Deterministic Finite-Difference Mirror Descent
  15. Vector field for deterministic finite-difference mirror descent
  16. ...and 23 more sections

Key Result

Theorem 2.1

Fix a constant stepsize $\eta>0$ and run the generalized mirror descent update $x_{j+1}=F_{\eta_j\Omega}(\mathbf{x}_j)$ (i.e., $\eta_j\equiv\eta$ for all $j$). If the objective function $f$ satisfies the relative smoothness condition RS, then for $j=1,\ldots,t-1$ we have

Theorems & Definitions (35)

  • Theorem 2.1: Monotonicity for a constant stepsize
  • Definition 2.2: Punctured-neighborhood generalized star-convexity
  • Theorem 2.3
  • Corollary 2.4
  • Lemma 3.1
  • Proof 1
  • Theorem 5.1: Refined step-size reduction for (trajectory-wise) relative smoothness
  • Theorem 6.1: Robust sufficient lower bound for $\alpha$
  • Theorem 6.2
  • Corollary 6.3
  • ...and 25 more