Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparsity via Sparse Group $k$-max Regularization

Qinghua Tao, Xiangming Xi, Jun Xu, Johan A. K. Suykens

TL;DR

This work tackles linear inverse problems with grouped sparsity by introducing sparse group $k$-max regularization, an appealing non-convex penalty that selectively promotes sparsity within each group by penalizing the smallest $d_i - k_i$ entries while imposing no extra magnitude constraints on the remaining entries. An iterative soft thresholding framework with a group $k$-max shrinkage operator is developed, along with local optimality conditions and a complexity analysis. The approach is validated on synthetic data and real-world datasets (Diabetes and Alcoholic EEG), showing enhanced group-wise and in-group sparsity with competitive accuracy and practical computation times. The proposed method provides a tunable alternative to convex relaxations that better approximates the $l_0$ norm and accommodates varying variable scales across groups, offering flexible, interpretable sparse solutions for structured inverse problems.

Abstract

For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with its convex counterparts. In this paper, we propose a novel and concise regularization, namely the sparse group $k$-max regularization, which can not only simultaneously enhance the group-wise and in-group sparsity, but also casts no additional restraints on the magnitude of variables in each group, which is especially important for variables at different scales, so that it approximate the $l_0$ norm more closely. We also establish an iterative soft thresholding algorithm with local optimality conditions and complexity analysis provided. Through numerical experiments on both synthetic and real-world datasets, we verify the effectiveness and flexibility of the proposed method.

Sparsity via Sparse Group $k$-max Regularization

TL;DR

This work tackles linear inverse problems with grouped sparsity by introducing sparse group -max regularization, an appealing non-convex penalty that selectively promotes sparsity within each group by penalizing the smallest entries while imposing no extra magnitude constraints on the remaining entries. An iterative soft thresholding framework with a group -max shrinkage operator is developed, along with local optimality conditions and a complexity analysis. The approach is validated on synthetic data and real-world datasets (Diabetes and Alcoholic EEG), showing enhanced group-wise and in-group sparsity with competitive accuracy and practical computation times. The proposed method provides a tunable alternative to convex relaxations that better approximates the norm and accommodates varying variable scales across groups, offering flexible, interpretable sparse solutions for structured inverse problems.

Abstract

For the linear inverse problem with sparsity constraints, the regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the regularization with its convex counterparts. In this paper, we propose a novel and concise regularization, namely the sparse group -max regularization, which can not only simultaneously enhance the group-wise and in-group sparsity, but also casts no additional restraints on the magnitude of variables in each group, which is especially important for variables at different scales, so that it approximate the norm more closely. We also establish an iterative soft thresholding algorithm with local optimality conditions and complexity analysis provided. Through numerical experiments on both synthetic and real-world datasets, we verify the effectiveness and flexibility of the proposed method.
Paper Structure (11 sections, 2 theorems, 25 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 11 sections, 2 theorems, 25 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. SPARSE GROUP $k$-MAX REGULARIZATION
  3. ALGORITHMS
  4. Modified Iterative Soft Thresholding Algorithm
  5. Optimality Analysis
  6. Complexity Analysis
  7. NUMERICAL EXPERIMENTS
  8. Synthetic Examples
  9. Diabetes Dataset
  10. Alcoholic EEG Dataset
  11. CONCLUSIONS AND FUTURE WORK

Key Result

Theorem 1

If $\exists \boldsymbol x^* \in \mathbb R^d$, for all possible $I_k^{\leq}(\boldsymbol u^*)$, where $\boldsymbol u^* = \boldsymbol x^*+\Phi^T(\boldsymbol{y}-\Phi \boldsymbol x^*)$, such that where $t_k$ is the $k$-th maximal absolute value of $\boldsymbol u^*$ and $t_k^+ = \min_{j\in I_k^+(\boldsymbol u^*)}\{\boldsymbol u^*(j)\}$, then $\boldsymbol x^*$ is locally optimal to (prob:group_kmax_op

Figures (2)

  • Figure 1: Visualization on the recovery of sparse signals with grouping characteristics using different regularizations. The horizontal axis denotes the index of entries in each $\boldsymbol{x}_i$ in the vector $\boldsymbol{x} = [\boldsymbol{x}_1^T, \cdots, \boldsymbol{x}_m^T]^T$, and the vertical axis represents the corresponding signal (variable) values.
  • Figure 2: Performance on the diabetes dataset.

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof