Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proximal gradient methods with inexact oracle of degree q for composite optimization

Yassine Nabou, Francois Glineur, Ion Necoara

TL;DR

The results show that better rates can be obtained both theoretically and in numerical simulations when q is large, and the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems.

Abstract

We introduce the concept of inexact first-order oracle of degree q for a possibly nonconvex and nonsmooth function, which naturally appears in the context of approximate gradient, weak level of smoothness and other situations. Our definition is less conservative than those found in the existing literature, and it can be viewed as an interpolation between fully exact and the existing inexact first-order oracle definitions. We analyze the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems. We derive complexity estimates and study the dependence between the accuracy of the oracle and the desired accuracy of the gradient or of the objective function. Our results show that better rates can be obtained both theoretically and in numerical simulations when q is large.

Proximal gradient methods with inexact oracle of degree q for composite optimization

TL;DR

The results show that better rates can be obtained both theoretically and in numerical simulations when q is large, and the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems.

Abstract

We introduce the concept of inexact first-order oracle of degree q for a possibly nonconvex and nonsmooth function, which naturally appears in the context of approximate gradient, weak level of smoothness and other situations. Our definition is less conservative than those found in the existing literature, and it can be viewed as an interpolation between fully exact and the existing inexact first-order oracle definitions. We analyze the convergence behavior of a (fast) inexact proximal gradient method using such an oracle for solving (non)convex composite minimization problems. We derive complexity estimates and study the dependence between the accuracy of the oracle and the desired accuracy of the gradient or of the objective function. Our results show that better rates can be obtained both theoretically and in numerical simulations when q is large.
Paper Structure (7 sections, 6 theorems, 70 equations, 1 figure, 3 algorithms)

This paper contains 7 sections, 6 theorems, 70 equations, 1 figure, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notations and preliminaries
  3. Inexact first-order oracle of degree $q$
  4. Inexact proximal gradient method
  5. Nonconvex convergence analysis
  6. Convex convergence analysis
  7. Numerical simulations

Key Result

Theorem 1

Let $F$ be a nonconvex function admitting a $(\delta_k,L_k)$-oracle of degree $q\in [0,2)$ at each iteration $k$, with $\delta_k \geq 0$ and $L_k > 0$ for all $k\geq 0$. Let $(x_k)_{k\geq 0}$ be generated by I-PGM and assume that $\alpha_k \leq \frac{1}{L_k + q\rho}$, for some arbitrary parameter $\

Figures (1)

  • Figure 1: Practical (dotted lines) and theoretical (full lines) performances of the I-PGM algorithm for different choices of $q$ and $\delta$, and with $R = 4$. Figure (b) represents a zoom of the left corner from Figure (a).

Theorems & Definitions (26)

  • Definition 1
  • Definition 2
  • Example 1
  • Remark 1
  • Example 2
  • Remark 2
  • Example 3
  • Remark 3
  • Example 4
  • Theorem 1
  • ...and 16 more