Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PL conditions do not guarantee convergence of gradient descent-ascent dynamics

Jean-Christophe Mourrat

Abstract

We give an example of a function satisfying a two-sided Polyak-Lojasiewicz condition but for which a gradient descent-ascent flow line fails to converge to the saddle point, circling around it instead.

PL conditions do not guarantee convergence of gradient descent-ascent dynamics

Abstract

We give an example of a function satisfying a two-sided Polyak-Lojasiewicz condition but for which a gradient descent-ascent flow line fails to converge to the saddle point, circling around it instead.
Paper Structure (3 sections, 7 theorems, 37 equations, 1 figure)

This paper contains 3 sections, 7 theorems, 37 equations, 1 figure.

Table of Contents

  1. Introduction
  2. General properties of PL functions
  3. Construction of the function

Key Result

Theorem 1.1

There exists a $\mathcal{C}^\infty$ function $f : [-1,1]^2 \to \mathbb{R}$ with a unique critical point at the origin, and a constant $C < +\infty$ such that for every $x, y \in [-1,1]^2$, and yet, for every $z(0)$ in some open subset of $[-1,1]^2$, the GDA flow given by e.def.grad.asc.desc is periodic.

Figures (1)

  • Figure 1: The flow lines with a color scale from dark blue to yellow are the level lines of the function $f$ we build for Theorem \ref{['t.main']} (the color scale indicates the magnitude of $\mathbf{v}$). The value of $f$ is not shown and is prescribed along the two orange lines according to \ref{['e.f.on.mclX']}. These two lines are also the set of points at which the level line of $f$ is horizontal or vertical (i.e., where $\partial_x f = 0$ or $\partial_y f = 0$). The red trajectory is a gradient descent-ascent flow line on $f$. The ellipses are the level lines of the quadratic forms inside the two occurrences of the function $\varphi$ in \ref{['e.def.v']}, for the values $1/2$ and $1$. In particular, the level lines of $f$ are tangent to the vector field given in \ref{['e.def.w']} on the region contained by every ellipse, and to the vector field given in \ref{['e.v.outside']} on the region that is outside of every ellipse.

Theorems & Definitions (14)

  • Theorem 1.1
  • Proposition 1.2: local convergence
  • Proposition 2.1: Criterion for PL condition
  • proof
  • proof : Proof of Proposition \ref{['p.loc.conv']}
  • Proposition 2.2: criterion for two-sided PL
  • proof
  • Proposition 3.1
  • proof
  • Proposition 3.2
  • ...and 4 more