Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proximal Iterative Hard Thresholding Algorithm for Sparse Group $\ell_0$-Regularized Optimization with Box Constraint

Yuge Ye, Qingna Li

TL;DR

This work addresses non-convex box-constrained optimization with simultaneous elementwise and group sparsity by deriving a closed-form proximal operator for the joint regularization and formulating a proximal iterative hard thresholding scheme (PIHT-SGB). It introduces the notions of $\tau$-stationary and support-optimal points to connect stationary conditions with minimizers and proves convergence to a local minimizer under standard smoothness/convexity assumptions, with finite support changes and a quantified iteration complexity. The method is validated through extensive numerical experiments on synthetic signal recovery and RGB image reconstruction, demonstrating robustness to initialization, the beneficial impact of box constraints, and clear gains from jointly exploiting $\ell_0$ and $\ell_{2,0}$ sparsity compared to PIHT and related approaches. Overall, PIHT-SGB provides an effective, scalable framework for solving non-convex, box-constrained sparse group optimization problems with practical relevance to imaging and signal processing.

Abstract

This paper investigates a general class of problems in which a lower bounded smooth convex function incorporating $\ell_{0}$ and $\ell_{2,0}$ regularization is minimized over a box constraint. Although such problems arise frequently in practical applications, their inherent non-convexity poses significant challenges for solution methods. In particular, we focus on the proximal operator associated with these regularizations, which incorporates both group-sparsity and element-wise sparsity terms. Besides, we introduce the concepts of $τ$-stationary point and support optimal (SO) point then analyze their relationship with the minimizer of the considered problem. Based on the proximal operator, we propose a novel proximal iterative hard thresholding algorithm to solve the problem. Furthermore, we establish the global convergence and the computational complexity analysis of the proposed method. Finally, extensive experiments demonstrate the effectiveness and efficiency of our method.

Proximal Iterative Hard Thresholding Algorithm for Sparse Group $\ell_0$-Regularized Optimization with Box Constraint

TL;DR

This work addresses non-convex box-constrained optimization with simultaneous elementwise and group sparsity by deriving a closed-form proximal operator for the joint regularization and formulating a proximal iterative hard thresholding scheme (PIHT-SGB). It introduces the notions of -stationary and support-optimal points to connect stationary conditions with minimizers and proves convergence to a local minimizer under standard smoothness/convexity assumptions, with finite support changes and a quantified iteration complexity. The method is validated through extensive numerical experiments on synthetic signal recovery and RGB image reconstruction, demonstrating robustness to initialization, the beneficial impact of box constraints, and clear gains from jointly exploiting and sparsity compared to PIHT and related approaches. Overall, PIHT-SGB provides an effective, scalable framework for solving non-convex, box-constrained sparse group optimization problems with practical relevance to imaging and signal processing.

Abstract

This paper investigates a general class of problems in which a lower bounded smooth convex function incorporating and regularization is minimized over a box constraint. Although such problems arise frequently in practical applications, their inherent non-convexity poses significant challenges for solution methods. In particular, we focus on the proximal operator associated with these regularizations, which incorporates both group-sparsity and element-wise sparsity terms. Besides, we introduce the concepts of -stationary point and support optimal (SO) point then analyze their relationship with the minimizer of the considered problem. Based on the proximal operator, we propose a novel proximal iterative hard thresholding algorithm to solve the problem. Furthermore, we establish the global convergence and the computational complexity analysis of the proposed method. Finally, extensive experiments demonstrate the effectiveness and efficiency of our method.

Paper Structure

This paper contains 20 sections, 14 theorems, 59 equations, 8 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Sparse Group Lasso
  3. Sparse Group $\ell_0$-Regularized Optimization
  4. Proximal Operator and Solution Method
  5. Proximal operator
  6. Solution method: PIHT-SGB
  7. Convergence and Complexity Analysis
  8. Related constrained problem
  9. Convergence and complexity analysis
  10. Numerical Results
  11. Signal recovery
  12. Different dimensions
  13. Different initializations
  14. Different boxes
  15. Comparison with PIHT
  16. ...and 5 more sections

Key Result

Lemma 2.1

lu2014iterative For $i=1,\cdots, n$, denote $s_{\tau}(x^k) = x^k - \tau \nabla f(x^k)$. The solution $x^{k+1}$ of eq-subproblem-l0 is given as follows:

Figures (8)

  • Figure 1: An illustration of signal restoration by PIHT-SGB with $n=2000 : 12000$ and $m=0.25n$
  • Figure 2: An illustration of iteration process in PIHT-SGB with $n=10000$ and $m=0.25n$.
  • Figure 3: Signal recovery by PIHT-SGB with different initial points and $m=0.25n$, $s = 0.05n$.
  • Figure 4: PIHT-SGB in different boxes.
  • Figure 5: Comparison with PIHT under different group sizes.
  • ...and 3 more figures

Theorems & Definitions (25)

  • Lemma 2.1
  • Proposition 2.1
  • Proposition 2.2
  • Theorem 2.3
  • proof
  • Definition 1
  • Proposition 2.4
  • proof
  • Definition 2
  • Definition 3
  • ...and 15 more