Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shadow-point Enhanced Inexact Accelerated Proximal Gradient Method with Preserved Convergence Guarantees

Lei Yang, Liwei Luo, Meixia Lin

TL;DR

SpinAPG addresses convex composite optimization $F(oldsymbol{x})=P(oldsymbol{x})+f(oldsymbol{x})$ by introducing a shadow-point enhanced inexact APG method that eliminates the need to frequently compute a feasible iterate while preserving convergence guarantees. The method combines an extrapolation framework with a two-point error criterion, allowing inexact gradient and proximal solves, and employs shadow points to certify progress without full feasibility. Theoretical results establish an ${ m O}(1/k^2)$ convergence rate for the objective gap and, under a tuned extrapolation schedule, a $o(1/k^2)$ rate with iterate convergence. Numerical experiments on a relaxation of the quadratic assignment problem demonstrate computational gains from bypassing explicit feasible-point computations, illustrating SpinAPG’s practical appeal for large-scale constrained problems.

Abstract

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using the inexact accelerated proximal gradient (APG) method. A key limitation of existing approaches is their reliance on feasible approximate solutions to subproblems, which is often computationally expensive or even unrealistic to obtain in practice. To address this limitation, we develop a shadow-point enhanced inexact accelerated proximal gradient method (SpinAPG), which can eliminate the feasibility requirement while preserving all desirable convergence properties of the APG method, including the iterate convergence and an $o(1/k^2)$ convergence rate for the objective function value, under suitable summable-error conditions. Our method also provides a more flexible and computationally efficient inexact framework for the APG method, with a fairly easy-to-implement error criterion. Finally, we demonstrate the practical advantages of our SpinAPG through numerical experiments on a relaxation of the quadratic assignment problem, showcasing its effectiveness while bypassing the explicit computation of a feasible point.

Shadow-point Enhanced Inexact Accelerated Proximal Gradient Method with Preserved Convergence Guarantees

TL;DR

SpinAPG addresses convex composite optimization by introducing a shadow-point enhanced inexact APG method that eliminates the need to frequently compute a feasible iterate while preserving convergence guarantees. The method combines an extrapolation framework with a two-point error criterion, allowing inexact gradient and proximal solves, and employs shadow points to certify progress without full feasibility. Theoretical results establish an convergence rate for the objective gap and, under a tuned extrapolation schedule, a rate with iterate convergence. Numerical experiments on a relaxation of the quadratic assignment problem demonstrate computational gains from bypassing explicit feasible-point computations, illustrating SpinAPG’s practical appeal for large-scale constrained problems.

Abstract

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using the inexact accelerated proximal gradient (APG) method. A key limitation of existing approaches is their reliance on feasible approximate solutions to subproblems, which is often computationally expensive or even unrealistic to obtain in practice. To address this limitation, we develop a shadow-point enhanced inexact accelerated proximal gradient method (SpinAPG), which can eliminate the feasibility requirement while preserving all desirable convergence properties of the APG method, including the iterate convergence and an convergence rate for the objective function value, under suitable summable-error conditions. Our method also provides a more flexible and computationally efficient inexact framework for the APG method, with a fairly easy-to-implement error criterion. Finally, we demonstrate the practical advantages of our SpinAPG through numerical experiments on a relaxation of the quadratic assignment problem, showcasing its effectiveness while bypassing the explicit computation of a feasible point.
Paper Structure (17 sections, 8 theorems, 96 equations, 1 figure, 2 tables, 1 algorithm)

This paper contains 17 sections, 8 theorems, 96 equations, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. A shadow-point enhanced inexact APG method
  3. Extrapolation parameter selection
  4. Shadow-point enhanced error criterion
  5. Overcoming feasibility challenges via shadow-points
  6. Convergence analysis
  7. The ${\cal O}(1/k^2)$ convergence rate
  8. The improved $o(1/k^2)$ convergence rate and the sequential convergence
  9. Numerical experiments
  10. Implementation of the inexact APG method
  11. Verification of error criteria in different inexact APG methods
  12. Comparison results
  13. Concluding remarks
  14. Supplementary proofs in Section \ref{['sec-convana']}
  15. Proof of Lemma \ref{['lem-thetasumbd']}
  16. ...and 2 more sections

Key Result

Lemma 3.1

Suppose that $\{\alpha_k\}_{k=0}^{\infty}\subseteq\mathbb{R}$ and $\{\gamma_k\}_{k=0}^{\infty}\subseteq\mathbb{R}_+$ are two sequences such that $\{\alpha_k\}$ is bounded from below, $\sum_{k=0}^{\infty} \gamma_k < \infty$, and $\alpha_{k+1} \leq \alpha_{k} + \gamma_k$ holds for all $k$. Then, $\{\a

Figures (1)

  • Figure 1: Average numerical results of SpinAPG (a), iAPG-SLB/AIFB (b), o-iFB (c), and I-FISTA (d) for solving problem \ref{['QAPprobrelax']} with $n$ ranging from 500 to 3000. All methods run for up to $15000$Ssncg iterations.

Theorems & Definitions (12)

  • Lemma 3.1: p1987introduction
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4: Approximate sufficient descent property
  • Lemma 3.5
  • proof
  • Theorem 3.1
  • proof
  • Lemma 3.6
  • proof
  • ...and 2 more