Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reachability of gradient descent

Cédric Josz, Wenqing Ouyang

Abstract

We show that gradient descent can converge to any local minimum of a smooth semi-algebraic function. This holds if the step sizes are nonsummable and sufficiently small. The same results hold for the subgradient method on locally Lipschitz semi-algebraic functions if the step size is constant.

Reachability of gradient descent

Abstract

We show that gradient descent can converge to any local minimum of a smooth semi-algebraic function. This holds if the step sizes are nonsummable and sufficiently small. The same results hold for the subgradient method on locally Lipschitz semi-algebraic functions if the step size is constant.

Paper Structure

This paper contains 8 sections, 8 theorems, 66 equations.

Table of Contents

  1. Introduction
  2. Notations
  3. Main body
  4. Continuous time
  5. Discrete time
  6. Funding
  7. Data availability
  8. Conflict of interest

Key Result

Theorem 1

Let $f:\mathbb{R}^ n \to \mathbb{R}$ be $C_L^ {1,1}$ definable and $\overline{x} \in \mathbb{R}^ n$ be a critical point of $f$ that is not a local maximum of $f$. Then where $\{x_k\}_{k\in\mathbb{N}}$ is defined by $x_{k+1} = x_k - \alpha_k \nabla f(x_k)$ for all $k\in\mathbb{N}$.

Theorems & Definitions (17)

  • Theorem 1
  • Theorem 2
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Proposition 1
  • proof
  • Lemma 3
  • proof
  • ...and 7 more