Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Complexity of Inexact Proximal Point Algorithm for minimizing convex functions with Holderian Growth

Andrei Pătraşcu, Paul Irofti

TL;DR

This work advances the nonasymptotic analysis of the Inexact Proximal Point Algorithm (IPPA) for convex minimization with Holderian growth, showing that proximal updates retain favorable growth properties via the Moreau envelope. It provides concrete iteration-complexity bounds that interpolate between weak sharp minima and quadratic growth, and it introduces a Restarted IPPA framework with provable efficiency gains. By coupling IPPA with an inner Proximal Subgradient Method, the authors derive overall complexity scales that adapt to problem smoothness and growth parameters, and they validate the approach with numerical experiments on robust L1 least squares, graph-regularized SVMs, and matrix completion. The results offer practical restart strategies that achieve fast convergence without requiring exact knowledge of growth parameters.

Abstract

Several decades ago the Proximal Point Algorithm (PPA) started to gain a long-lasting attraction for both abstract operator theory and numerical optimization communities. Even in modern applications, researchers still use proximal minimization theory to design scalable algorithms that overcome nonsmoothness. Remarkable works as \cite{Fer:91,Ber:82constrained,Ber:89parallel,Tom:11} established tight relations between the convergence behaviour of PPA and the regularity of the objective function. In this manuscript we derive nonasymptotic iteration complexity of exact and inexact PPA to minimize convex functions under $γ-$Holderian growth: $\BigO{\log(1/ε)}$ (for $γ\in [1,2]$) and $\BigO{1/ε^{γ- 2}}$ (for $γ> 2$). In particular, we recover well-known results on PPA: finite convergence for sharp minima and linear convergence for quadratic growth, even under presence of deterministic noise. Moreover, when a simple Proximal Subgradient Method is recurrently called as an inner routine for computing each IPPA iterate, novel computational complexity bounds are obtained for Restarting Inexact PPA. Our numerical tests show improvements over existing restarting versions of the Subgradient Method.

Complexity of Inexact Proximal Point Algorithm for minimizing convex functions with Holderian Growth

TL;DR

This work advances the nonasymptotic analysis of the Inexact Proximal Point Algorithm (IPPA) for convex minimization with Holderian growth, showing that proximal updates retain favorable growth properties via the Moreau envelope. It provides concrete iteration-complexity bounds that interpolate between weak sharp minima and quadratic growth, and it introduces a Restarted IPPA framework with provable efficiency gains. By coupling IPPA with an inner Proximal Subgradient Method, the authors derive overall complexity scales that adapt to problem smoothness and growth parameters, and they validate the approach with numerical experiments on robust L1 least squares, graph-regularized SVMs, and matrix completion. The results offer practical restart strategies that achieve fast convergence without requiring exact knowledge of growth parameters.

Abstract

Several decades ago the Proximal Point Algorithm (PPA) started to gain a long-lasting attraction for both abstract operator theory and numerical optimization communities. Even in modern applications, researchers still use proximal minimization theory to design scalable algorithms that overcome nonsmoothness. Remarkable works as \cite{Fer:91,Ber:82constrained,Ber:89parallel,Tom:11} established tight relations between the convergence behaviour of PPA and the regularity of the objective function. In this manuscript we derive nonasymptotic iteration complexity of exact and inexact PPA to minimize convex functions under Holderian growth: (for ) and (for ). In particular, we recover well-known results on PPA: finite convergence for sharp minima and linear convergence for quadratic growth, even under presence of deterministic noise. Moreover, when a simple Proximal Subgradient Method is recurrently called as an inner routine for computing each IPPA iterate, novel computational complexity bounds are obtained for Restarting Inexact PPA. Our numerical tests show improvements over existing restarting versions of the Subgradient Method.

Paper Structure

This paper contains 13 sections, 16 theorems, 60 equations, 5 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Notations and preliminaries
  3. Holderian growth and Moreau envelopes
  4. Inexact Proximal Point algorithm
  5. Iteration complexity of IPPA
  6. Restarted Inexact Proximal Point Algorithm
  7. Inner Proximal Subgradient Method routine
  8. Numerical simulations
  9. Robust $\ell_1$ Least Squares
  10. Graph Support Vector Machines
  11. Matrix Completion for Movie Recommendation
  12. Appendix
  13. Appendix B

Key Result

Lemma 2.1

Let $F$ be a convex function and let $\gamma-$(HG) hold. Then the Moreau envelope $F_{\mu}$ satisfies the relations presented below. $(i)$ Let $\gamma = 1$ WSM: where $H_{\tau}(s) = $ is the Huber function. $(ii)$ Let $\gamma = 2$: $(iii)$ For all $\gamma \ge 1$: where $\varphi(\gamma) = \min_{\lambda \in [0,1]} \lambda^{\gamma} + (1-\lambda)^2$.

Figures (5)

  • Figure 1: Total number of inner iterations needed for various parametrizations. (a) Varying $\rho$. (b) Varied problem dimensions where we set $m=n$. (c) Varying $\tau$
  • Figure 2: GraphSVM experiments on synthetic data, first row, on the 20newsgroups data-set, second row, and with $M=I_n$ on the third row. (a), (c), (e): Objective error evolution across inner iterations. (b), (d), (f): Objective error evolution across inner iterations measured in CPU time. The error is displayed in logarithmic scale.
  • Figure 3: GraphSVM: Total number of inner iterations needed for various parametrizations. (a) Varying $\rho$. (b) Varied problem dimensions where we set $m=n$. (c) Varied $\tau$.
  • Figure 4: Sparse $\ell_1$-SVM: (a) Objective error evolution across inner iterations. (b) Zoomed objective error evolution.
  • Figure 5: Matrix completion: (a) Objective error evolution across inner iterations. (b) Objective error evolution across inner iterations measured in CPU time.

Theorems & Definitions (20)

  • Lemma 2.1
  • Remark 1
  • Lemma 3.1
  • Theorem 4.1
  • Corollary 4.2
  • Corollary 4.3
  • Corollary 4.4
  • Theorem 5.1
  • Remark 2
  • Theorem 6.1
  • ...and 10 more