Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Characterizations of inexact proximal operators

Guillaume Lauga, Samuel Vaiter

TL;DR

The paper addresses the challenge of using inexact proximal operators within non-smooth optimization, especially when penalties are non-convex. It introduces six approximations (types (a)–(f)) to the proximal operator, develops systematic criteria for their qualitative accuracy, regularity, and admissibility, and studies how these approximations interact with proximal-point and splitting algorithms. The results show that, under weakly convex penalties with $\rho<1$, convergence to an approximate solution is possible even when errors are non-summable, and provide explicit bounds for proximal-point and splitting schemes. The findings offer practical guidance for plug-and-play and learned-prior methods, showing which approximation strategies are most conducive to convergence and how to quantify the impact of inexactness on the attained solution.

Abstract

Proximal operators are now ubiquitous in non-smooth optimization. Since their introduction in the seminal work of Moreau, many papers have shown their effectiveness on a wide variety of problems, culminating in their use to construct convergent deep learning methods. The characterization of these operators for non-convex penalties was completed recently in [Gribonval et al, A characterization of proximity operators, 2020]. In this paper, we propose to follow this line of work by characterizing inexact proximal operators, thus providing an answer to what constitutes a good approximation of these operators. We propose several definitions of approximations and discuss their regularity, approximation power, and their fixed points. Equipped with these characterizations, we investigate the convergence of proximal algorithms in the presence of errors that may be non-summable and/or non-vanishing. In particular, we look at the proximal point algorithm, and at the forward-backward, Peaceman-Rachford and Douglas-Rachford algorithms when we minimize the sum of a weakly convex function (whose proximal operator is approximated) and a strongly convex function.

Characterizations of inexact proximal operators

TL;DR

The paper addresses the challenge of using inexact proximal operators within non-smooth optimization, especially when penalties are non-convex. It introduces six approximations (types (a)–(f)) to the proximal operator, develops systematic criteria for their qualitative accuracy, regularity, and admissibility, and studies how these approximations interact with proximal-point and splitting algorithms. The results show that, under weakly convex penalties with , convergence to an approximate solution is possible even when errors are non-summable, and provide explicit bounds for proximal-point and splitting schemes. The findings offer practical guidance for plug-and-play and learned-prior methods, showing which approximation strategies are most conducive to convergence and how to quantify the impact of inexactness on the attained solution.

Abstract

Proximal operators are now ubiquitous in non-smooth optimization. Since their introduction in the seminal work of Moreau, many papers have shown their effectiveness on a wide variety of problems, culminating in their use to construct convergent deep learning methods. The characterization of these operators for non-convex penalties was completed recently in [Gribonval et al, A characterization of proximity operators, 2020]. In this paper, we propose to follow this line of work by characterizing inexact proximal operators, thus providing an answer to what constitutes a good approximation of these operators. We propose several definitions of approximations and discuss their regularity, approximation power, and their fixed points. Equipped with these characterizations, we investigate the convergence of proximal algorithms in the presence of errors that may be non-summable and/or non-vanishing. In particular, we look at the proximal point algorithm, and at the forward-backward, Peaceman-Rachford and Douglas-Rachford algorithms when we minimize the sum of a weakly convex function (whose proximal operator is approximated) and a strongly convex function.
Paper Structure (53 sections, 46 theorems, 229 equations, 1 figure, 1 table)

This paper contains 53 sections, 46 theorems, 229 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Context of the paper.
  3. Related works.
  4. Contributions.
  5. Notations.
  6. Optimization background
  7. Subdifferential and approximate subdifferential
  8. Proximal operators and Moreau envelope
  9. The proximal operator.
  10. Error bounds
  11. Convergence of fixed point iterations
  12. Convex setting
  13. Non-convex setting.
  14. Approximations of the proximal operator
  15. Approximation of type (a)
  16. ...and 38 more sections

Key Result

Proposition 1

moreau1965proximite, gribonval2020characterization. A function $z : \mathbb{R}^N \to \mathbb{R}^N$ defined everywhere is the proximal operator of a proper convex l.s.c. function $\phi : \mathbb{R}^N \to \overline{\mathbb{R}}$ if, and only if, the following conditions hold jointly:

Figures (1)

  • Figure 1: Contractivity of the operators, with respect to the Lipschitz constant of $\mathrm{g}$ and the strong convexity of $f$, when the continuity is exact and varying $\mu/L_f$ for the optimal value of $\tau$ryu2016primer.

Theorems & Definitions (102)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Definition 9
  • Proposition 1
  • ...and 92 more