Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence analysis of dynamical systems for optimization by an improved Lyapunov framework

Atsushi Tabei, Ken'ichiro Tanaka

Abstract

We study the convergence analysis of continuous-time dynamical systems associated with optimization methods for strongly convex functions. Recent works have proposed systematic constructions of Lyapunov functions for such analysis, while also revealing limitations of the Lyapunov analysis. Aujol--Dossal--Rondepierre (2023) have proposed a technique to address this issue by reorganizing Lyapunov functions so as to evaluate a quantity $f(x(t)) - f_* - g(t)\|x(t)-x_*\|^2$ rather than $f(x(t)) - f_*$. By combining this technique with our computer-assisted framework to discover Lyapunov functions, we develop an improved method that reproduces an existing convergence rate or yields better rates than previous studies.

Convergence analysis of dynamical systems for optimization by an improved Lyapunov framework

Abstract

We study the convergence analysis of continuous-time dynamical systems associated with optimization methods for strongly convex functions. Recent works have proposed systematic constructions of Lyapunov functions for such analysis, while also revealing limitations of the Lyapunov analysis. Aujol--Dossal--Rondepierre (2023) have proposed a technique to address this issue by reorganizing Lyapunov functions so as to evaluate a quantity rather than . By combining this technique with our computer-assisted framework to discover Lyapunov functions, we develop an improved method that reproduces an existing convergence rate or yields better rates than previous studies.

Paper Structure

This paper contains 4 sections, 9 theorems, 22 equations.

Table of Contents

  1. Introduction
  2. Dynamical System $\ddot{x} + \frac{r}{t}\dot{x} + \nabla f = 0$
  3. Dynamical System $\ddot{x} + \frac{r}{t^{\alpha}}\dot{x} + \nabla f = 0$
  4. Conclusion

Key Result

Lemma 1

Suppose that a Lyapunov function $\mathcal{E}$ satisfies for some $\mathcal{F}(t)$ and $\gamma(t)$. Then the system satisfies

Theorems & Definitions (11)

  • Definition 1: Lyapunov function
  • Lemma 1
  • Definition 2: $\mu$-strongly convex
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Lemma 5
  • Lemma 6
  • Lemma 7
  • ...and 1 more