Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The sparse Kaczmarz method with surrogate hyperplane for the regularized basis pursuit problem

Ze Wang, Jun-Feng Yin, Ji-Chen Zhao

TL;DR

The paper tackles sparse recovery via the regularized basis pursuit problem and introduces surrogate hyperplane sparse Kaczmarz methods to accelerate convergence. It develops residual-based and partial residual-based implementations, grounded in Bregman-distance analysis of the $f(x)=\lambda\|x\|_1+\frac{1}{2}\|x\|_2^2$ objective, and proves linear convergence with explicit contraction factors. The authors provide detailed theoretical guarantees and extensive numerical experiments on random Gaussian and SuiteSparse matrices, as well as image restoration tasks, showing faster convergence than traditional randomized sparse Kaczmarz approaches and enhanced reconstruction quality. The work offers practical, scalable algorithms for sparse recovery in compressed sensing and imaging with strong convergence guarantees and demonstrated empirical efficiency.

Abstract

The Sparse Kaczmarz method is a famous and widely used iterative method for solving the regularized basis pursuit problem. A general scheme of the surrogate hyperplane sparse Kaczmarz method is proposed. In particular, a class of residual-based surrogate hyperplane sparse Kaczmarz method is introduced and the implementations are well discussed. Their convergence theories are proved and the linear convergence rates are studied and compared in details. Numerical experiments verify the efficiency of the proposed methods.

The sparse Kaczmarz method with surrogate hyperplane for the regularized basis pursuit problem

TL;DR

The paper tackles sparse recovery via the regularized basis pursuit problem and introduces surrogate hyperplane sparse Kaczmarz methods to accelerate convergence. It develops residual-based and partial residual-based implementations, grounded in Bregman-distance analysis of the objective, and proves linear convergence with explicit contraction factors. The authors provide detailed theoretical guarantees and extensive numerical experiments on random Gaussian and SuiteSparse matrices, as well as image restoration tasks, showing faster convergence than traditional randomized sparse Kaczmarz approaches and enhanced reconstruction quality. The work offers practical, scalable algorithms for sparse recovery in compressed sensing and imaging with strong convergence guarantees and demonstrated empirical efficiency.

Abstract

The Sparse Kaczmarz method is a famous and widely used iterative method for solving the regularized basis pursuit problem. A general scheme of the surrogate hyperplane sparse Kaczmarz method is proposed. In particular, a class of residual-based surrogate hyperplane sparse Kaczmarz method is introduced and the implementations are well discussed. Their convergence theories are proved and the linear convergence rates are studied and compared in details. Numerical experiments verify the efficiency of the proposed methods.
Paper Structure (5 sections, 6 theorems, 53 equations, 5 figures, 4 tables, 3 algorithms)

This paper contains 5 sections, 6 theorems, 53 equations, 5 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. The surrogate hyperplane sparse Kaczmarz method
  3. The Residual-based Surrogate hyperplane sparse Kaczmarz method
  4. Numerical Experiments
  5. Conclusion

Key Result

Lemma 2.1

lorenz2014linearized Let $f$ be continuous and strongly convex with constant $\alpha>0$. For all $x,y\in \mathbb{R}^{n}$ and $x^*\in \partial f(x)$, it is satisfied that and $D_f^{x^{*}}(x, y)=0$ if and only if $x=y$.

Figures (5)

  • Figure 1: Convergence curves for random overdetermined matrices.
  • Figure 2: Convergence curves for random underdetermined matrices.
  • Figure 3: Convergence curves for matrices from the SuiteSparse Matrix Collection.
  • Figure 4: Comparison of reconstruction results for 2-D seismic travel-time tomography.
  • Figure 5: Comparison of restoration results for the number ranging from 0 to 9.

Theorems & Definitions (10)

  • Lemma 2.1
  • Lemma 2.2
  • Lemma 2.3
  • proof
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Corollary 3.1
  • proof