Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Resolution Inertial Dynamics with Time-Rescaled Gradients for Nonsmooth Convex Optimization

Manh Hung Le, Andrea Simonetto

Abstract

We study nonsmooth convex minimization through a continuous-time dynamical system that can be seen as a high-resolution ODE of Nesterov Accelerated Gradient (NAG) adapted to the nonsmooth case. We apply a time-varying Moreau envelope smoothing to a proper convex lower semicontinuous objective function and introduce a controlled time-rescaling of the gradient, coupled with a Hessian-driven damping term, leading to our proposed inertial dynamic. We provide a well-posedness result for this dynamical system, and construct a Lyapunov energy function capturing the combined effects of inertia, damping, and smoothing. For an appropriate scaling, the energy dissipates and yields fast decay of the objective function and gradient, stabilization of velocities, and weak convergence of trajectories to minimizers under mild assumptions. Conceptually, the system is a nonsmooth high-resolution model of Nesterov's method that clarifies how time-varying smoothing and time rescaling jointly govern acceleration and stability. We further extend the framework to the setting of maximally monotone operators, for which we propose and analyze a corresponding dynamical system and establish analogous convergence results. We also present numerical experiments illustrating the effect of the main parameters and comparing the proposed system with several benchmark dynamics.

High-Resolution Inertial Dynamics with Time-Rescaled Gradients for Nonsmooth Convex Optimization

Abstract

We study nonsmooth convex minimization through a continuous-time dynamical system that can be seen as a high-resolution ODE of Nesterov Accelerated Gradient (NAG) adapted to the nonsmooth case. We apply a time-varying Moreau envelope smoothing to a proper convex lower semicontinuous objective function and introduce a controlled time-rescaling of the gradient, coupled with a Hessian-driven damping term, leading to our proposed inertial dynamic. We provide a well-posedness result for this dynamical system, and construct a Lyapunov energy function capturing the combined effects of inertia, damping, and smoothing. For an appropriate scaling, the energy dissipates and yields fast decay of the objective function and gradient, stabilization of velocities, and weak convergence of trajectories to minimizers under mild assumptions. Conceptually, the system is a nonsmooth high-resolution model of Nesterov's method that clarifies how time-varying smoothing and time rescaling jointly govern acceleration and stability. We further extend the framework to the setting of maximally monotone operators, for which we propose and analyze a corresponding dynamical system and establish analogous convergence results. We also present numerical experiments illustrating the effect of the main parameters and comparing the proposed system with several benchmark dynamics.

Paper Structure

This paper contains 28 sections, 10 theorems, 249 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem and motivation
  3. High-resolution ODE models
  4. Time-rescaled dynamics
  5. Moreau envelope smoothing for nonsmooth problems
  6. Our dynamics: a nonsmooth high-resolution approach with time-rescaled gradients
  7. Paper outline
  8. Wellposedness of $(\text{NS-HR})_{\alpha,\beta}$
  9. First-order reformulation of $(\text{NS-HR})_{\alpha,\beta}$
  10. Existence and uniqueness of a solution trajectory of $(\text{NS-HR})_{\alpha,\beta}$
  11. Convergence properties of $(\text{NS-HR})_{\alpha,\beta}$
  12. Extension to the setting of maximally monotone operators
  13. Discussion on optimization vs. maximally monotone operators
  14. Discretization of the (NS-HR)$_{\alpha,\beta}$ dynamic
  15. Second-order discretization
  16. ...and 13 more sections

Key Result

Theorem 2.1

Suppose $\delta:[t_0,+\infty ) \to (0,+\infty)$ and $\gamma:[t_0,+\infty ) \to (0,+\infty)$ are continuously differentiable functions. Let $(x_0,v_0,y_0) \in \mathcal{H}^3$ satisfying Then, the following are equivalent

Figures (3)

  • Figure 1: Influence of the parameter $\beta$ on the behavior of $(\mathrm{NS\text{-}HR})_{\alpha,\beta}$.
  • Figure 2: Influence of the parameter $\alpha$ on the behavior of $(\mathrm{NS\text{-}HR})_{\alpha,\beta}$.
  • Figure 3: Comparison of $(\mathrm{NS\text{-}HR})_{\alpha,\beta}$ with four benchmark models: the two baseline systems, the Attouch--László dynamic, and the Bot–Karapetyants dynamic.

Theorems & Definitions (15)

  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Remark 3.1
  • Lemma 3.1
  • Theorem 3.1
  • Lemma 3.2: Opial's lemma
  • Lemma 3.3
  • Remark 3.2
  • Remark 3.3
  • ...and 5 more