Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Iterates of Nesterov's Accelerated Algorithm Converge in The Critical Regimes

Radu Ioan Bot, Jalal Fadili, Dang-Khoa Nguyen

TL;DR

This work proves that the iterates of the accelerated Nesterov method in the critical regime ($\alpha=3$) converge weakly to a minimizer for convex $L_{\nabla f}$-smooth functions on real Hilbert spaces, resolving a longstanding conjecture by Attouch and coauthors. It develops a Lyapunov-energy framework with $W_k$ and $\mathcal{E}_{z,k}$, plus an ergodic averaging representation, to establish boundedness and weak convergence of the iterates, and extends the results to the proximal-gradient setting (FISTA). The analysis aligns with Ryu's recent continuous-time convergence results and remains valid for smooth+nonsmooth problems, providing a robust theoretical foundation for accelerated methods in the critical regime. The findings solidify convergence properties of Nesterov-type acceleration in infinite-dimensional spaces and undergird practical use in proximal-gradient algorithms.

Abstract

In this paper, we prove that the iterates of the accelerated Nesterov's algorithm in the critical regime do converge in the weak topology to a global minimizer of an $L$-smooth function in a real Hilbert space, hence answering positively a conjecture posed by H. Attouch and co-authors a decade ago. This result is the algorithmic case of a very recent result on the continuous-time system posted by E. Ryu on X, with assistance from ChatGPT.

The Iterates of Nesterov's Accelerated Algorithm Converge in The Critical Regimes

TL;DR

This work proves that the iterates of the accelerated Nesterov method in the critical regime () converge weakly to a minimizer for convex -smooth functions on real Hilbert spaces, resolving a longstanding conjecture by Attouch and coauthors. It develops a Lyapunov-energy framework with and , plus an ergodic averaging representation, to establish boundedness and weak convergence of the iterates, and extends the results to the proximal-gradient setting (FISTA). The analysis aligns with Ryu's recent continuous-time convergence results and remains valid for smooth+nonsmooth problems, providing a robust theoretical foundation for accelerated methods in the critical regime. The findings solidify convergence properties of Nesterov-type acceleration in infinite-dimensional spaces and undergird practical use in proximal-gradient algorithms.

Abstract

In this paper, we prove that the iterates of the accelerated Nesterov's algorithm in the critical regime do converge in the weak topology to a global minimizer of an -smooth function in a real Hilbert space, hence answering positively a conjecture posed by H. Attouch and co-authors a decade ago. This result is the algorithmic case of a very recent result on the continuous-time system posted by E. Ryu on X, with assistance from ChatGPT.
Paper Structure (8 sections, 5 theorems, 50 equations, 1 algorithm)

This paper contains 8 sections, 5 theorems, 50 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Contributions
  4. Convergence proof
  5. Properties of the sequence $\left( t_{k} \right)_{k \geq 1}$
  6. Lyapunov analysis and boundedness of the iterates
  7. Weak convergence of the iterates
  8. Accelerated proximal-gradient algorithm

Key Result

Lemma 1

Let $\left( t_{k} \right)_{k \geq 1}$ be a sequence defined according to defi:t. For every $k \geq 1$, it holds:

Theorems & Definitions (12)

  • Remark 1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Remark 2
  • Proposition 1
  • Proposition 2
  • proof
  • Theorem 1
  • ...and 2 more