Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lyapunov stability of the Euler method

Cédric Josz

TL;DR

This work addresses stability of Euler discretizations for set-valued dynamical systems by coupling a continuous-time potential $f$ with a discrete-time Lyapunov function $g$ and developing a comprehensive d-stability framework. It introduces $p$-d-Lyapunov and $(p,q)$-d-Lyapunov notions, along with tracking and descent lemmas, to bridge discrete Euler updates with continuous trajectories and establish stability and asymptotic convergence criteria. The authors extend stability from points to sets and define attractors within this discrete-time conservative-field context, with practical criteria for verifying d-Lyapunov properties via conserved quantities, Verdier conditions, and metric subregularity. Applications to subdifferential dynamics and the role of conservation and symmetry reveal connections to flat minima and implicit regularization, offering a principled approach to understanding discretized optimization dynamics and the emergence of stable flat regions in training.

Abstract

We extend the Lyapunov stability criterion to Euler discretizations of set-valued dynamical systems. It relies on a pair of Lyapunov functions, one in continuous time and one in discrete time. In the context of optimization, this yields sufficient conditions for normalized gradient descent to converge to a region containing the flat minima.

Lyapunov stability of the Euler method

TL;DR

This work addresses stability of Euler discretizations for set-valued dynamical systems by coupling a continuous-time potential with a discrete-time Lyapunov function and developing a comprehensive d-stability framework. It introduces -d-Lyapunov and -d-Lyapunov notions, along with tracking and descent lemmas, to bridge discrete Euler updates with continuous trajectories and establish stability and asymptotic convergence criteria. The authors extend stability from points to sets and define attractors within this discrete-time conservative-field context, with practical criteria for verifying d-Lyapunov properties via conserved quantities, Verdier conditions, and metric subregularity. Applications to subdifferential dynamics and the role of conservation and symmetry reveal connections to flat minima and implicit regularization, offering a principled approach to understanding discretized optimization dynamics and the emergence of stable flat regions in training.

Abstract

We extend the Lyapunov stability criterion to Euler discretizations of set-valued dynamical systems. It relies on a pair of Lyapunov functions, one in continuous time and one in discrete time. In the context of optimization, this yields sufficient conditions for normalized gradient descent to converge to a region containing the flat minima.

Paper Structure

This paper contains 11 sections, 18 theorems, 86 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Theory
  4. Lyapunov functions
  5. Stable points
  6. Stable sets
  7. Attractors
  8. Applications
  9. Subdifferential dynamics
  10. Conservation, flatness, and symmetry
  11. Examples

Key Result

Theorem 1

Let $\overline{x}\in\mathbb{R}^n$ and $F:\mathbb{R}^n\rightrightarrows\mathbb{R}^n$. Suppose: Then $\overline x$ is stable.

Figures (3)

  • Figure 1: $f(x,y)=y^2+x^2y^4$ and $g(x,y)=|x|$. We plot the contours of both $f$ and $g$ on the $xy$-plane. The trajectory of the red points is obtained by running normalized gradient method with stepsize $1/(k+1)^{1/6}$ from initial point $(2.5,0.01)$.
  • Figure 2: $f(x,y)=(x^2-y)^2$ and $g(x,y)=x^2\exp(4y)$. We plot the contours of both $f$ and $\log(g)$ on the $xy$-plane. The trajectory of the red points is obtained by running normalized gradient method with stepsize $0.1/(k+1)^{1/6}$ from initial point $(0.9,0.7)$.
  • Figure 3: $f(x,y)=(2x^2+y^2-1)^2$ and $g(x,y)=|x|/y^2$. We plot the contours of both $f$ and $g$ on the $xy$-plane. The trajectory of the points is obtained by running normalized gradient method with stepsize $1/(k+1)^{1/6}$ from initial point $(0.8,0.2)$.

Theorems & Definitions (55)

  • Theorem 1: Lyapunov stability criterion
  • Theorem 2
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • Definition 1
  • ...and 45 more