Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$\ell_{1\text{-}2}$ Regularization for Sparse Optimization: Consistency and Global Convergence

Yaohua Hu, Hao Wang, Xiaoqi Yang

TL;DR

Preliminary numerical results show that the proposed algorithms can approach the ground true sparse solution and significantly enhance the sparsity recovery capability, compared with the popular sparse optimization algorithms in the literature.

Abstract

The $\ell_{1\text{-}2}$ regularization method has a strong sparsity promoting capability in approaching sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. This paper aims to investigate the consistency theory and global convergent algorithms for the $\ell_{1\text{-}2}$ regularization problem. In the theoretical aspect, we introduce a notion of restricted eigenvalue condition relative to the $\ell_{1\text{-}2}$ penalty, and employ it to establish an oracle property and a recovery bound for the global solution of the $\ell_{1\text{-}2}$ regularization problem. In the algorithmic aspect, we propose two types of iterative thresholding algorithms with the truncation technique and the continuation technique, respectively, to solve the $\ell_{1\text{-}2}$ regularization problem. Moreover, under the assumption of the well-known restricted isometry property, we establish the convergence of the proposed algorithms to the ground true sparse solution within a tolerance relevant to the noise level and the recovery bound. Preliminary numerical results show that our proposed algorithms can approach the ground true sparse solution and significantly enhance the sparsity recovery capability, compared with the popular sparse optimization algorithms in the literature.

$\ell_{1\text{-}2}$ Regularization for Sparse Optimization: Consistency and Global Convergence

TL;DR

Preliminary numerical results show that the proposed algorithms can approach the ground true sparse solution and significantly enhance the sparsity recovery capability, compared with the popular sparse optimization algorithms in the literature.

Abstract

The regularization method has a strong sparsity promoting capability in approaching sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. This paper aims to investigate the consistency theory and global convergent algorithms for the regularization problem. In the theoretical aspect, we introduce a notion of restricted eigenvalue condition relative to the penalty, and employ it to establish an oracle property and a recovery bound for the global solution of the regularization problem. In the algorithmic aspect, we propose two types of iterative thresholding algorithms with the truncation technique and the continuation technique, respectively, to solve the regularization problem. Moreover, under the assumption of the well-known restricted isometry property, we establish the convergence of the proposed algorithms to the ground true sparse solution within a tolerance relevant to the noise level and the recovery bound. Preliminary numerical results show that our proposed algorithms can approach the ground true sparse solution and significantly enhance the sparsity recovery capability, compared with the popular sparse optimization algorithms in the literature.
Paper Structure (10 sections, 10 theorems, 122 equations, 3 figures, 1 table, 2 algorithms)

This paper contains 10 sections, 10 theorems, 122 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Consistency theory
  3. Regularity conditions
  4. Oracle property and recovery bound
  5. Iterative Thresholding Algorithms
  6. $\ell_{1\text{-}2}$ thresholding operator
  7. Iterative thresholding algorithm with truncation technique
  8. Iterative thresholding algorithm with continuation technique
  9. Numerical experiments
  10. Proof of Proposition \ref{['prop-REC-sufficient']}

Key Result

Proposition 2.1

$A\in {\mathbb{R}}^{m\times n}$ satisfies the $\ell_{1\text{-}2}$-${\rm REC}(s,t)$ provided that one of the following conditions holds:

Figures (3)

  • Figure 1: Numerical results of ITAC and ITAT with different parameters.
  • Figure 2: Convergence behavior of ITAs for sparse optimization.
  • Figure 3: Successful recovery rates of ITAs and algorithms for sparse optimization.

Theorems & Definitions (27)

  • Definition 2.1
  • Definition 2.2: RIC CandesTao05
  • Definition 2.3: MIC Donoho2001Uncertainty
  • Definition 2.4: SEC Donoho2006Most
  • Proposition 2.1
  • Lemma 2.1: HuJMLR17
  • Theorem 2.1
  • proof
  • Remark 2.1
  • Lemma 3.1
  • ...and 17 more