Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Splitting Proximal Point Algorithms for the Sum of Prox-Convex Functions

Jose de Brito, Felipe Lara, Tran Van Thang

TL;DR

This work extends splitting proximal point methods to minimize a finite sum of prox-convex functions $f(x)=\sum_{i=1}^N f_i(x)$ over a closed set $K$, where each $f_i$ is Lipschitz and prox-convex. It introduces two variants: a deterministic permuted-order method and a stochastic random-order method, both leveraging proximal operators of individual components. The authors prove global convergence for the deterministic algorithm and almost-sure convergence for the stochastic variant under standard diminishing stepsizes, using a descent-type analysis and a supermartingale argument, respectively. Numerical experiments on nonconvex quadratic problems illustrate practical efficiency and confirm theoretical results, highlighting trade-offs between deterministic and stochastic updates in terms of robustness, per-iteration cost, and scalability.

Abstract

This paper addresses the minimization of a finite sum of prox-convex functions under Lipschitz continuity of each component. We propose two variants of the splitting proximal point algorithms proposed in \cite{Bacak,Bertsekas}: one deterministic with a fixed update order, and one stochastic with random sampling, and we extend them from convex to prox-convex functions. We prove global convergence for both methods under standard stepsize a\-ssump\-tions, with almost sure convergence for the stochastic variant via supermartingale theory. Numerical experiments with nonconvex quadratic functions illustrate the efficiency of the proposed methods and support the theoretical results.

Splitting Proximal Point Algorithms for the Sum of Prox-Convex Functions

TL;DR

This work extends splitting proximal point methods to minimize a finite sum of prox-convex functions over a closed set , where each is Lipschitz and prox-convex. It introduces two variants: a deterministic permuted-order method and a stochastic random-order method, both leveraging proximal operators of individual components. The authors prove global convergence for the deterministic algorithm and almost-sure convergence for the stochastic variant under standard diminishing stepsizes, using a descent-type analysis and a supermartingale argument, respectively. Numerical experiments on nonconvex quadratic problems illustrate practical efficiency and confirm theoretical results, highlighting trade-offs between deterministic and stochastic updates in terms of robustness, per-iteration cost, and scalability.

Abstract

This paper addresses the minimization of a finite sum of prox-convex functions under Lipschitz continuity of each component. We propose two variants of the splitting proximal point algorithms proposed in \cite{Bacak,Bertsekas}: one deterministic with a fixed update order, and one stochastic with random sampling, and we extend them from convex to prox-convex functions. We prove global convergence for both methods under standard stepsize a\-ssump\-tions, with almost sure convergence for the stochastic variant via supermartingale theory. Numerical experiments with nonconvex quadratic functions illustrate the efficiency of the proposed methods and support the theoretical results.
Paper Structure (10 sections, 8 theorems, 58 equations, 7 figures, 2 tables)

This paper contains 10 sections, 8 theorems, 58 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Basic Definitions
  3. Splitting Proximal Point Algorithms
  4. Permuted-order Version
  5. Method 1 (Permuted-order version).
  6. Random Order Version.
  7. Method 2 (Stochastic Version).
  8. Numerical Experiments
  9. Conclusions
  10. Declarations

Key Result

Lemma 2.1

(see T2024) Let $\mathbb R^n = \prod^{N}_{i=1} \mathbb{R}^{n_{i}}$, where $\sum^{N}_{i=1} n_{i} = n$ and $n_i \geq 1$, with $i=1,\ldots,N$. Then for any $x \in \mathbb R^n$ we have $x=(x^1,\ldots,x^{N})$ where $x^i \in \mathbb R^{n_i}$ for every $i=1,\ldots,N$. For each $i\in \{1,\ldots,N\}$, let $C

Figures (7)

  • Figure 1: Convergence of $\log_{10}(Error)$ of Method 1 and Method 2 with respect to the number of iterations and CPU time(s), where $n=100$ and $\epsilon=10^{-10}$
  • Figure 2: Convergence of $\log_{10}(Error)$ of Method 1 and Method 2 with respect to the number of iterations and CPU time(s), where $n=100$ and $\epsilon=10^{-12}$
  • Figure 3: Convergence of $\log_{10}(Error)$ of Method 1 and Method 2 with respect to the number of iterations and CPU time(s), where $n=100$ and $\epsilon=10^{-13}$
  • Figure 4: Convergence of $\log_{10}(Error)$ of Method 1 and Method 2 with respect to the number of iterations and CPU time(s), where $n=50$ and $\epsilon=10^{-10}$
  • Figure 5: Convergence of $\log_{10}(Error)$ of Method 1 and Method 2 with respect to the number of iterations and CPU time(s), where $n=70$ and $\epsilon=10^{-10}$
  • ...and 2 more figures

Theorems & Definitions (20)

  • Definition 2.1
  • Example 2.1
  • Lemma 2.1
  • Lemma 2.2
  • Remark 2.1
  • Lemma 2.3
  • proof
  • Proposition 3.1
  • proof
  • Remark 3.1
  • ...and 10 more