Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On a fixed-point continuation method for a convex optimization problem

Jean-Baptiste Fest, Tommi Heikkilä, Ignace Loris, Ségolène Martin, Luca Ratti, Simone Rebegoldi, Gesa Sarnighausen

Abstract

We consider a variation of the classical proximal-gradient algorithm for the iterative minimization of a cost function consisting of a sum of two terms, one smooth and the other prox-simple, and whose relative weight is determined by a penalty parameter. This so-called fixed-point continuation method allows one to approximate the problem's trade-off curve, i.e. to compute the minimizers of the cost function for a whole range of values of the penalty parameter at once. The algorithm is shown to converge, and a rate of convergence of the cost function is also derived. Furthermore, it is shown that this method is related to iterative algorithms constructed on the basis of the $ε$-subdifferential of the prox-simple term. Some numerical examples are provided.

On a fixed-point continuation method for a convex optimization problem

Abstract

We consider a variation of the classical proximal-gradient algorithm for the iterative minimization of a cost function consisting of a sum of two terms, one smooth and the other prox-simple, and whose relative weight is determined by a penalty parameter. This so-called fixed-point continuation method allows one to approximate the problem's trade-off curve, i.e. to compute the minimizers of the cost function for a whole range of values of the penalty parameter at once. The algorithm is shown to converge, and a rate of convergence of the cost function is also derived. Furthermore, it is shown that this method is related to iterative algorithms constructed on the basis of the -subdifferential of the prox-simple term. Some numerical examples are provided.
Paper Structure (6 sections, 5 theorems, 45 equations, 2 figures)

This paper contains 6 sections, 5 theorems, 45 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Trade-off curve
  3. Convergence analysis
  4. Numerical experiments
  5. Approximating the trade-off curve
  6. Conclusions

Key Result

lemma 1

If $f:\mathbb{R}^d \rightarrow \mathbb{R}$ is convex with Lipschitz continuous gradient ($L$) then $\frac{1}{L}\nabla f$ is firmly non expansive:

Figures (2)

  • Figure 1: Left: Graphical representation of the so-called trade-off curve and its relation to the penalty parameter $\lambda$. Right: Path (in the $g-f$-plane) of two different iterative optimization algorithms applied to the same instance of problem (\ref{['eq:problem']}) and starting from the same initial point. The black dotted path is special with respect to a generic path, as the former path approximately samples the trade-off curve (i.e. intermediate iterates have some interest) and the latter produces uninteresting intermediate iterates.
  • Figure 2: Paths of the different sequences in the $g-f$ plane compared to the trade-off curve

Theorems & Definitions (13)

  • proof
  • remark 1
  • lemma 1
  • proof
  • lemma 2
  • proof
  • lemma 3
  • proof
  • theorem 1
  • proof
  • ...and 3 more