Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A proximal-gradient inertial algorithm with Tikhonov regularization: strong convergence to the minimal norm solution

Szilárd Csaba László

TL;DR

The paper tackles the convex composite optimization problem $\inf_{x} f(x)+g(x)$ by introducing a proximal-gradient inertial method augmented with two Tikhonov regularization terms. Under carefully chosen sequences for the inertial parameter and regularization weights, the iterates converge strongly to the minimum-norm minimizer of $f+g$, while achieving fast decay rates for the objective gap and related quantities. It provides explicit rate results in terms of the parameter choices, including regimes where $O(k^{-p})$ or $O(k^{-2})$ decay is obtained, and validates the theory with numerical experiments showing the necessity of both regularization terms. The work offers a practical, robust algorithm with strong convergence guarantees and potential extensions to more complex structured problems.

Abstract

We investigate the strong convergence properties of a proximal-gradient inertial algorithm with two Tikhonov regularization terms in connection to the minimization problem of the sum of a convex lower semi-continuous function $f$ and a smooth convex function $g$. For the appropriate setting of the parameters we provide strong convergence of the generated sequence $(x_k)$ to the minimum norm minimizer of our objective function $f+g$. Further, we obtain fast convergence to zero of the objective function values in a generated sequence but also for the discrete velocity and the sub-gradient of the objective function. We also show that for another settings of the parameters the optimal rate of order $\mathcal{O}(k^{-2})$ for the potential energy $(f+g)(x_k)-\min(f+g)$ can be obtained.

A proximal-gradient inertial algorithm with Tikhonov regularization: strong convergence to the minimal norm solution

TL;DR

The paper tackles the convex composite optimization problem by introducing a proximal-gradient inertial method augmented with two Tikhonov regularization terms. Under carefully chosen sequences for the inertial parameter and regularization weights, the iterates converge strongly to the minimum-norm minimizer of , while achieving fast decay rates for the objective gap and related quantities. It provides explicit rate results in terms of the parameter choices, including regimes where or decay is obtained, and validates the theory with numerical experiments showing the necessity of both regularization terms. The work offers a practical, robust algorithm with strong convergence guarantees and potential extensions to more complex structured problems.

Abstract

We investigate the strong convergence properties of a proximal-gradient inertial algorithm with two Tikhonov regularization terms in connection to the minimization problem of the sum of a convex lower semi-continuous function and a smooth convex function . For the appropriate setting of the parameters we provide strong convergence of the generated sequence to the minimum norm minimizer of our objective function . Further, we obtain fast convergence to zero of the objective function values in a generated sequence but also for the discrete velocity and the sub-gradient of the objective function. We also show that for another settings of the parameters the optimal rate of order for the potential energy can be obtained.
Paper Structure (13 sections, 12 theorems, 124 equations, 2 figures)

This paper contains 13 sections, 12 theorems, 124 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Motivation and related works
  3. On condition \ref{['condT']} and a model result
  4. The outline of the paper
  5. Strong convergence and fast rates
  6. Particular choices of $t_k$ and $\epsilon_k$.
  7. Numerical experiments
  8. Conclusions, perspectives
  9. Declarations
  10. Appendix
  11. The descent lemma and its consequences
  12. Three pillars that sustain our results
  13. On the Tikhonov regularization techniques

Key Result

Theorem 1

For $s<\frac{1}{L}$ let $(x_k)_{k\ge 0},\,(y_k)_{k\ge 1}$ be the sequences generated by Algorithm tdiscgen1.

Figures (2)

  • Figure 1: From a numerical point of view $q=1$ is not the best choice. Nevertheless, in case $q=1$\ref{['tdiscgen1']} might have a better behaviour than FISTA.
  • Figure 2: Renouncing to one of the Tikhonov regularization terms in \ref{['tdiscgen1']} there is no convergence to the minimum norm solution anymore.

Theorems & Definitions (26)

  • Theorem 1
  • Remark 2
  • Remark 3
  • Theorem 4
  • proof
  • Remark 5
  • Theorem 6
  • proof
  • Theorem 7
  • proof
  • ...and 16 more