Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Block Bregman Projection with Polyak-like Stepsize for Possibly Inconsistent Convex Feasibility Problems

Lu Zhang, Hongzhen Chen, Hongxia Wang, Hui Zhang

Abstract

Stochastic projection algorithms for solving convex feasibility problems (CFPs) have attracted considerable attention due to their broad applicability. In this paper, we propose a unified stochastic bilevel reformulation for possibly inconsistent CFPs that combines proximity function minimization and structural regularization, leading to a feasible bilevel model with a unique and stable regularized solution. From the algorithmic perspective, we develop the stochastic block Bregman projection method with Polyak-like and projective stepsizes, which not only subsumes several recent stochastic projection algorithms but also induces new schemes tailored to specific problems. Moreover, we establish ergodic sublinear convergence rates for the expected inner function, as well as linear convergence in expectation to the inner minimizer set under a Bregman distance growth condition. In particular, the proposed Polyak-like stepsize ensures exact convergence in expectation for possibly inconsistent CFPs. Finally, numerical experiments demonstrate the effectiveness of the proposed method and its robustness to noise.

Stochastic Block Bregman Projection with Polyak-like Stepsize for Possibly Inconsistent Convex Feasibility Problems

Abstract

Stochastic projection algorithms for solving convex feasibility problems (CFPs) have attracted considerable attention due to their broad applicability. In this paper, we propose a unified stochastic bilevel reformulation for possibly inconsistent CFPs that combines proximity function minimization and structural regularization, leading to a feasible bilevel model with a unique and stable regularized solution. From the algorithmic perspective, we develop the stochastic block Bregman projection method with Polyak-like and projective stepsizes, which not only subsumes several recent stochastic projection algorithms but also induces new schemes tailored to specific problems. Moreover, we establish ergodic sublinear convergence rates for the expected inner function, as well as linear convergence in expectation to the inner minimizer set under a Bregman distance growth condition. In particular, the proposed Polyak-like stepsize ensures exact convergence in expectation for possibly inconsistent CFPs. Finally, numerical experiments demonstrate the effectiveness of the proposed method and its robustness to noise.

Paper Structure

This paper contains 31 sections, 17 theorems, 121 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Organization
  4. Preliminaries
  5. Notation
  6. Convex analysis tools
  7. Bregman distance tools
  8. Probability results
  9. The proposed algorithmic framework
  10. Stochastic reformulation
  11. The proposed method
  12. The choice of stepsize $t_k$
  13. Decreasing mirror stochastic Polyak stepsize via SMD
  14. Projective stepsize via cut-and-project
  15. Convergence analysis
  16. ...and 16 more sections

Key Result

Lemma 1

Let $\psi:\mathbb{R}^n\rightarrow \mathbb{R}$ be $\mu$-strongly convex. Then its conjugate $\psi^*$ is differentiable and has a Lipschitz continuous gradient with modulus $1/\mu$, i.e., Moreover, the conditions $\psi(x)+\psi^*(x^*)=\langle x,x^*\rangle,~x^*\in\partial \psi(x)$, and $x\in\partial \psi^*(x^*)$ are equivalent.

Figures (6)

  • Figure 1: The performance of SBBP with DecmSPS for $\gamma_b=100$ and varying $c\in\{0.1,0.2,0.5,1\}$
  • Figure 2: The performance of SBBP with DecmSPS under $c=0.2$ and $\gamma_b\in\{2,20,50,100\}$
  • Figure 3: The performance of SBBP with different stepsizes under $\gamma_b=100$ and $\tau\in\{20,100,200,\tau^*\}$.
  • Figure 4: The convergence of SBP and SBBP with different stepsizes for several matrix sizes
  • Figure 5: The performance of RBP and SBBP with different stepsizes with $\sigma=0.01$
  • ...and 1 more figures

Theorems & Definitions (39)

  • Lemma 1
  • Definition 1
  • Lemma 2: Lemma 2.6, schopfer2019linear
  • Definition 2
  • Lemma 3: Lemma 2.2, lorenz2014linearized
  • Lemma 4: Lemma 2.4, lorenz2014linearized
  • Example 1
  • Lemma 5
  • Theorem 1
  • Lemma 6: Basic descent lemma
  • ...and 29 more