Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An efficient sieving based secant method for sparse optimization problems with least-squares constraints

Qian Li, Defeng Sun, Yancheng Yuan

TL;DR

An adaptive sieving technique is incorporated into the secant method to effectively reduce the dimension of the level-set subproblems for computing the value of $\varphi(\cdot)$ when the objective function in the constrained problem is a polyhedral gauge function.

Abstract

In this paper, we propose an efficient sieving based secant method to address the computational challenges of solving sparse optimization problems with least-squares constraints. A level-set method has been introduced in [X. Li, D.F. Sun, and K.-C. Toh, SIAM J. Optim., 28 (2018), pp. 1842--1866] that solves these problems by using the bisection method to find a root of a univariate nonsmooth equation $\varphi(λ) = \varrho$ for some $\varrho > 0$, where $\varphi(\cdot)$ is the value function computed by a solution of the corresponding regularized least-squares optimization problem. When the objective function in the constrained problem is a polyhedral gauge function, we prove that (i) for any positive integer $k$, $\varphi(\cdot)$ is piecewise $C^k$ in an open interval containing the solution $λ^*$ to the equation $\varphi(λ) = \varrho$; (ii) the Clarke Jacobian of $\varphi(\cdot)$ is always positive. These results allow us to establish the essential ingredients of the fast convergence rates of the secant method. Moreover, an adaptive sieving technique is incorporated into the secant method to effectively reduce the dimension of the level-set subproblems for computing the value of $\varphi(\cdot)$. The high efficiency of the proposed algorithm is demonstrated by extensive numerical results.

An efficient sieving based secant method for sparse optimization problems with least-squares constraints

TL;DR

An adaptive sieving technique is incorporated into the secant method to effectively reduce the dimension of the level-set subproblems for computing the value of when the objective function in the constrained problem is a polyhedral gauge function.

Abstract

In this paper, we propose an efficient sieving based secant method to address the computational challenges of solving sparse optimization problems with least-squares constraints. A level-set method has been introduced in [X. Li, D.F. Sun, and K.-C. Toh, SIAM J. Optim., 28 (2018), pp. 1842--1866] that solves these problems by using the bisection method to find a root of a univariate nonsmooth equation for some , where is the value function computed by a solution of the corresponding regularized least-squares optimization problem. When the objective function in the constrained problem is a polyhedral gauge function, we prove that (i) for any positive integer , is piecewise in an open interval containing the solution to the equation ; (ii) the Clarke Jacobian of is always positive. These results allow us to establish the essential ingredients of the fast convergence rates of the secant method. Moreover, an adaptive sieving technique is incorporated into the secant method to effectively reduce the dimension of the level-set subproblems for computing the value of . The high efficiency of the proposed algorithm is demonstrated by extensive numerical results.
Paper Structure (15 sections, 16 theorems, 106 equations, 6 figures, 8 tables, 4 algorithms)

This paper contains 15 sections, 16 theorems, 106 equations, 6 figures, 8 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Properties of the value function $\varphi(\cdot)$
  4. The HS-Jacobian of $\varphi(\cdot)$ for polyhedral gauge functions $p(\cdot)$
  5. A secant method for \ref{['eq: main-prob']}
  6. A fast convergent secant method for semismooth equations
  7. A globally convergent secant method for \ref{['eq: main-prob']}
  8. An adaptive sieving based secant method for \ref{['eq: main-prob']}
  9. An adaptive sieving technique for sparse optimization problems
  10. An adaptive sieving based secant method for \ref{['eq: main-prob']}
  11. Numerical experiments
  12. The $\ell_1$ penalized problems with least-squares constraints
  13. The sorted $\ell_1$ penalized problems with least-squares constraints
  14. A group lasso penalized problems with least-squares constraints
  15. Conclusion

Key Result

Lemma 2.2

\newlabellemma: lateral-derivative-approx0 Assume that $f: \mathbb{R} \to \mathbb{R}$ is semismooth at $\bar{x} \in \mathbb{R}$. Denote the lateral derivatives of $f$ at $\bar{x}$ by

Figures (6)

  • Figure 2: The performance of BMOP and SMOP in generating a solution path for \ref{['eq: main-prob']} with the $\ell_1$ regularization and stopping tolerance $10^{-6}$.
  • Figure 3: The ratio of the computation time between BMOP to the computation time of SMOP in solving \ref{['eq: main-prob']} with the sorted $\ell_1$ regularization.
  • Figure 4: The ratio of the computation time between BMOP to the computation time of SMOP in solving \ref{['eq: main-prob']} with the group lasso regularization.
  • Figure : Test I
  • Figure : Test I
  • ...and 1 more figures

Theorems & Definitions (28)

  • Definition 2.1
  • Lemma 2.2
  • Proposition 3.1
  • Proof 1
  • Proposition 3.2
  • Proposition 3.3
  • Proof 2
  • Corollary 3.4
  • Proposition 3.5
  • Proof 3
  • ...and 18 more