Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles

Amir Ali Farzin, Iman Shames

TL;DR

The application of a zeroth-order scheme for minimising Polyak-Lojasewicz functions is considered and the convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case is presented.

Abstract

The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are presented. The theoretical results are demonstrated via numerical examples.

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles

TL;DR

The application of a zeroth-order scheme for minimising Polyak-Lojasewicz functions is considered and the convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case is presented.

Abstract

The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are presented. The theoretical results are demonstrated via numerical examples.
Paper Structure (9 sections, 7 theorems, 48 equations, 2 figures, 2 algorithms)

This paper contains 9 sections, 7 theorems, 48 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problems of Interest
  4. Main Results
  5. Unconstrained Problem
  6. Constrained Problem
  7. Numerical Examples
  8. Conclusion and Future Research Directions
  9. Auxiliary Lemmas

Key Result

Lemma 1

PL inequality implies that every stationary point of the function is a global minimum.

Figures (2)

  • Figure 1: The evolution of the empirical mean of $f(\hat{x}_k)$ and the calculated upper bound versus the number of iterations in Scenario 1.
  • Figure 2: The evolution of $f(x_k)$ versus the number of iterations. Note that in this case $f(x^\ast)=0$ in Scenario 2.

Theorems & Definitions (21)

  • Definition 1: $C^{1,1}$ Functions
  • Remark 1
  • Definition 2: PL Functions polyak1964gradient
  • Lemma 1
  • proof
  • Definition 3: Proximal PL Functions
  • Definition 4: The big O-notation
  • Theorem 1
  • proof
  • Remark 2: $\mathrm{RS}_{\mu}$ Complexity, Parameter Selection, and Solution Error Bound
  • ...and 11 more