Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Hardness of the $L_1-L_2$ Regularization Problem

Yuyuan Ouyang, Kyle Yates

TL;DR

This paper establishes that both constrained and unconstrained L1-L2 minimization formulations for sparse reconstruction are NP-hard. It achieves this via polynomial-time reductions from the partition problem, constructing instances where solving the L1-L2 problems would solve Partition. The hardness results hold across parameter regimes: tau in (0, sqrt(2)) for the constrained case and lambda in (0, 2) (as well as lambda >= 2 via scaling) for the unconstrained case, and remain true even when nonnegativity constraints are added. The findings highlight fundamental computational barriers to exact L1-L2 sparse recovery and motivate future work on tractable subclasses and approximation methods within the broader DC programming framework.

Abstract

The sparse linear reconstruction problem is a core problem in signal processing which aims to recover sparse solutions to linear systems. The original problem regularized by the total number of nonzero components (also known as $L_0$ regularization) is well-known to be NP-hard. The relaxation of the $L_0$ regularization by using the $L_1$ norm offers a convex reformulation, but is only exact under certain conditions (e.g., restricted isometry property) which might be NP-hard to verify. To overcome the computational hardness of the $L_0$ regularization problem while providing tighter results than the $L_1$ relaxation, several alternate optimization problems have been proposed to find sparse solutions. One such problem is the $L_1-L_2$ minimization problem, which is to minimize the difference of the $L_1$ and $L_2$ norms subject to linear constraints. This paper proves that solving the $L_1-L_2$ minimization problem is NP-hard. Specifically, we prove that it is NP-hard to minimize the $L_1-L_2$ regularization function subject to linear constraints. Moreover, it is also NP-hard to solve the unconstrained formulation that minimizes the sum of a least squares term and the $L_1-L_2$ regularization function. Furthermore, restricting the feasible set to a smaller one by adding nonnegative constraints does not change the NP-hardness nature of the problems.

On the Hardness of the $L_1-L_2$ Regularization Problem

TL;DR

This paper establishes that both constrained and unconstrained L1-L2 minimization formulations for sparse reconstruction are NP-hard. It achieves this via polynomial-time reductions from the partition problem, constructing instances where solving the L1-L2 problems would solve Partition. The hardness results hold across parameter regimes: tau in (0, sqrt(2)) for the constrained case and lambda in (0, 2) (as well as lambda >= 2 via scaling) for the unconstrained case, and remain true even when nonnegativity constraints are added. The findings highlight fundamental computational barriers to exact L1-L2 sparse recovery and motivate future work on tractable subclasses and approximation methods within the broader DC programming framework.

Abstract

The sparse linear reconstruction problem is a core problem in signal processing which aims to recover sparse solutions to linear systems. The original problem regularized by the total number of nonzero components (also known as regularization) is well-known to be NP-hard. The relaxation of the regularization by using the norm offers a convex reformulation, but is only exact under certain conditions (e.g., restricted isometry property) which might be NP-hard to verify. To overcome the computational hardness of the regularization problem while providing tighter results than the relaxation, several alternate optimization problems have been proposed to find sparse solutions. One such problem is the minimization problem, which is to minimize the difference of the and norms subject to linear constraints. This paper proves that solving the minimization problem is NP-hard. Specifically, we prove that it is NP-hard to minimize the regularization function subject to linear constraints. Moreover, it is also NP-hard to solve the unconstrained formulation that minimizes the sum of a least squares term and the regularization function. Furthermore, restricting the feasible set to a smaller one by adding nonnegative constraints does not change the NP-hardness nature of the problems.

Paper Structure

This paper contains 9 sections, 12 theorems, 63 equations.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Notation
  4. NP-Hardness of Constrained $L_1-L_2$ Minimization
  5. NP-Hardness of Unconstrained $L_1-L_2$ Minimization
  6. NP-Hardness of the (\ref{['eq:NUP']}) instance when $\lambda\in(0,2)$
  7. NP-Hardness of the \ref{['eq:UP']} instance when $\lambda\in (0,2)$
  8. NP-hardness when $\lambda \geq 2$
  9. Concluding Remarks

Key Result

proposition thmcounterproposition

The problem of determining the solvability of the partition problem concerning multiset $S=\{a_1,\ldots,a_m\}$ is equivalent to determining whether the optimal objective value of the following optimization problem is $-m$: Here we denote $\textup{a}:=(a_1,\ldots,a_m)^\top$.

Theorems & Definitions (24)

  • proposition thmcounterproposition
  • proof
  • theorem 1
  • proof
  • lemma thmcounterlemma
  • proof
  • theorem 2
  • proof
  • proposition thmcounterproposition
  • proof
  • ...and 14 more