Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A fast iterative thresholding and support-and-scale shrinking algorithm (fits3) for non-lipschitz group sparse optimization (i): the case of least-squares fidelity

Yanan Zhao, Qiaoli Dong, Yufei Zhao, Chunlin Wu

TL;DR

The paper targets non-convex, non-Lipschitz group sparse optimization with least-squares fidelity by introducing FITS$^3$, a fast iterative thresholding algorithm that integrates thresholding, group support-and-scale shrinking, linearization, and extrapolation. It proves global convergence to a stationary point of the objective $\mathcal{E}(x)=\frac{1}{2}\|A x-b\|_2^2+\alpha\sum_{g_i\in\mathcal{G}}\psi(\|x_{g_i}\|_p)$ under a KŁ framework and a suitable inexact subproblem condition, while ensuring finite-group support stabilization. The method avoids solving linear systems, with closed-form updates for $p=1$ and $p=2$, enabling efficient scaling to large problems; experiments show competitive recovery accuracy and substantial CPU-time savings against recent non-Lipschitz group-sparse solvers. These results suggest FITS$^3$ as a practical tool for large-scale, structured sparsity recovery tasks in signal processing and related areas.

Abstract

We consider to design a new efficient and easy-to-implement algorithm to solve a general group sparse optimization model with a class of non-convex non-Lipschitz regularizations, named as fast iterative thresholding and support-and-scale shrinking algorithm (FITS3). In this paper we focus on the case of a least-squares fidelity. FITS3 is designed from a lower bound theory of such models and by integrating thresholding operation, linearization and extrapolation techniques. The FITS3 has two advantages. Firstly, it is quite efficient and especially suitable for large-scale problems, because it adopts support-and-scale shrinking and does not need to solve any linear or nonlinear system. For two important special cases, the FITS3 contains only simple calculations like matrix-vector multiplication and soft thresholding. Secondly, the FITS3 algorithm has a sequence convergence guarantee under proper assumptions. The numerical experiments and comparisons to recent existing non-Lipschitz group recovery algorithms demonstrate that, the proposed FITS3 achieves similar recovery accuracies, but costs only around a half of the CPU time by the second fastest compared algorithm for median or large-scale problems.

A fast iterative thresholding and support-and-scale shrinking algorithm (fits3) for non-lipschitz group sparse optimization (i): the case of least-squares fidelity

TL;DR

The paper targets non-convex, non-Lipschitz group sparse optimization with least-squares fidelity by introducing FITS, a fast iterative thresholding algorithm that integrates thresholding, group support-and-scale shrinking, linearization, and extrapolation. It proves global convergence to a stationary point of the objective under a KŁ framework and a suitable inexact subproblem condition, while ensuring finite-group support stabilization. The method avoids solving linear systems, with closed-form updates for and , enabling efficient scaling to large problems; experiments show competitive recovery accuracy and substantial CPU-time savings against recent non-Lipschitz group-sparse solvers. These results suggest FITS as a practical tool for large-scale, structured sparsity recovery tasks in signal processing and related areas.

Abstract

We consider to design a new efficient and easy-to-implement algorithm to solve a general group sparse optimization model with a class of non-convex non-Lipschitz regularizations, named as fast iterative thresholding and support-and-scale shrinking algorithm (FITS3). In this paper we focus on the case of a least-squares fidelity. FITS3 is designed from a lower bound theory of such models and by integrating thresholding operation, linearization and extrapolation techniques. The FITS3 has two advantages. Firstly, it is quite efficient and especially suitable for large-scale problems, because it adopts support-and-scale shrinking and does not need to solve any linear or nonlinear system. For two important special cases, the FITS3 contains only simple calculations like matrix-vector multiplication and soft thresholding. Secondly, the FITS3 algorithm has a sequence convergence guarantee under proper assumptions. The numerical experiments and comparisons to recent existing non-Lipschitz group recovery algorithms demonstrate that, the proposed FITS3 achieves similar recovery accuracies, but costs only around a half of the CPU time by the second fastest compared algorithm for median or large-scale problems.
Paper Structure (12 sections, 8 theorems, 90 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 12 sections, 8 theorems, 90 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation and preliminary
  3. Algorithm
  4. Convergence analysis
  5. Numerical experiments
  6. Test on Algorithm \ref{['alg_accelerated_ITS3']} with $p=2$
  7. The convergence and extrapolation effect of FITS$^3$
  8. Choice of $q$
  9. Adaptability to different group sizes and different types of $A$
  10. Comparisons
  11. Test on Algorithm \ref{['alg_accelerated_ITS3']} with $p=1$
  12. Conclusions

Key Result

Lemma 2.1

(hu2017group) \newlabellemma_inequa For any $w\in\mathbb{R}^{d}$, $0<\gamma_1\leq \gamma_2$, we have the following inequality

Figures (3)

  • Figure 5.1: Test on FITS$^3$ with $p=2$. (a) The recovery by FITS$^3$. (b)(c)(d)(e) The convergence justification of FITS$^3$. (f) The speed up effect of extrapolation.
  • Figure 5.2: Success rate tests and comparisons on FITS$^3$ with $p=2$. (a)(b)(c) The success rates of FITS$^3$ for group sparse minimization models with different $q$, different group sizes and different types of measurement matrix $A$, respectively. (d) Comparison on success rates between non-Lipschitz group sparse recovery algorithms and ADMM-GL algorithm.
  • Figure 5.3: Signal recovery by FITS$^3$ with $p=1$ and $p=2$ for the signal recovery problems with intra-group sparsity structures. (a)(b) The intra-group sparsity level $\underline{s}/l=6/32$. (c)(d) The intra-group sparsity level $\underline{s}/l=9/32$.

Theorems & Definitions (20)

  • Remark 2.1
  • Lemma 2.1
  • Lemma 2.2
  • Theorem 3.1
  • proof
  • Remark 3.1
  • Remark 3.2
  • Remark 4.1
  • Lemma 4.1
  • proof
  • ...and 10 more