Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Relative Inexact Proximal Gradient Method with an Explicit Linesearch

Yunier Bello-Cruz, Max L. N. Gonçalves, Jefferson G. Melo, Cassandra Mohr

TL;DR

This paper tackles the composite convex optimization problem F(x)=f(x)+g(x) without requiring Lipschitz continuity of ∇f. It introduces IPG-ELS, an inexact proximal gradient method with an explicit line search on the smooth component and a relative, controlled inexact proximal subproblem. The authors prove convergence to the optimal set and establish iteration-complexity results, including rates for η-approximate stationary solutions, under general assumptions and Lipschitz conditions where applicable. Numerical experiments on CUR-like factorization demonstrate that IPG-ELS can outperform exact-proximal and fixed-stepsize variants by effectively balancing subproblem accuracy with a data-driven line search. The approach provides a practically efficient and theoretically sound tool for large-scale, nonsmooth convex optimization problems in which proximal operators are expensive or unavailable in closed form.

Abstract

This paper presents and investigates an inexact proximal gradient method for solving composite convex optimization problems characterized by an objective function composed of a sum of a full-domain differentiable convex function and a non-differentiable convex function. We introduce an explicit line search applied specifically to the differentiable component of the objective function, requiring only a relative inexact solution of the proximal subproblem per iteration. We prove the convergence of the sequence generated by our scheme and establish its iteration complexity, considering both the functional values and a residual associated with first-order stationary solutions. Additionally, we provide numerical experiments to illustrate the practical efficacy of our method.

A Relative Inexact Proximal Gradient Method with an Explicit Linesearch

TL;DR

This paper tackles the composite convex optimization problem F(x)=f(x)+g(x) without requiring Lipschitz continuity of ∇f. It introduces IPG-ELS, an inexact proximal gradient method with an explicit line search on the smooth component and a relative, controlled inexact proximal subproblem. The authors prove convergence to the optimal set and establish iteration-complexity results, including rates for η-approximate stationary solutions, under general assumptions and Lipschitz conditions where applicable. Numerical experiments on CUR-like factorization demonstrate that IPG-ELS can outperform exact-proximal and fixed-stepsize variants by effectively balancing subproblem accuracy with a data-driven line search. The approach provides a practically efficient and theoretically sound tool for large-scale, nonsmooth convex optimization problems in which proximal operators are expensive or unavailable in closed form.

Abstract

This paper presents and investigates an inexact proximal gradient method for solving composite convex optimization problems characterized by an objective function composed of a sum of a full-domain differentiable convex function and a non-differentiable convex function. We introduce an explicit line search applied specifically to the differentiable component of the objective function, requiring only a relative inexact solution of the proximal subproblem per iteration. We prove the convergence of the sequence generated by our scheme and establish its iteration complexity, considering both the functional values and a residual associated with first-order stationary solutions. Additionally, we provide numerical experiments to illustrate the practical efficacy of our method.
Paper Structure (10 sections, 13 theorems, 119 equations, 2 tables)

This paper contains 10 sections, 13 theorems, 119 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Inexact proximal solutions
  4. Inexact proximal solutions for separable functions
  5. Inexact projection computed by the Frank-Wolfe method
  6. Related concepts of inexact proximal solutions
  7. Inexact Proximal Gradient Method
  8. Convergence and Complexity Analysis of the IPG-ELS Method
  9. Numerical Experiments
  10. Conclusions

Key Result

proposition thmcounterproposition

Let $(\varepsilon_k,x_k,v_k)_{k\in\mathbbm{N}}\subseteq \mathbbm{R}_+\times \mathbbm{E}\times \mathbbm{E}$ be a sequence converging to $(\varepsilon,x,v)$ with $v_k\in \partial_{\varepsilon_k} g(x_k)$ for all $k\in\mathbbm{N}$. Then, $v\in \partial_\varepsilon g(x)$.

Theorems & Definitions (19)

  • proposition thmcounterproposition: Closed Graph Property
  • proposition thmcounterproposition: Transportation Formula
  • definition thmcounterdefinition: $\eta$-Approximate Stationary Solution
  • definition thmcounterdefinition: Quasi-Fejér Convergence
  • lemma thmcounterlemma: Quasi-Féjer Convergence Properties
  • lemma thmcounterlemma
  • definition thmcounterdefinition
  • remark thmcounterremark: The Explicit Lineasearch
  • lemma thmcounterlemma: Iteration Inequality Condition
  • lemma thmcounterlemma: Stopping at a Solution
  • ...and 9 more