Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Iteratively Reweighted Method for Sparse Optimization on Nonconvex $\ell_{p}$ Ball

Hao Wang, Xiangyu Yang, Wei Jiang

TL;DR

An iteratively reweighted method is proposed, which solves a sequence of weighted $\ell_{1}$-ball projection subproblems, and it is proved that the generated iterates converge to a first-order stationary point.

Abstract

This paper is intended to solve the nonconvex $\ell_{p}$-ball constrained nonlinear optimization problems. An iteratively reweighted method is proposed, which solves a sequence of weighted $\ell_{1}$-ball projection subproblems. At each iteration, the next iterate is obtained by moving along the negative gradient with a stepsize and then projecting the resulted point onto the weighted $\ell_{1}$ ball to approximate the $\ell_{p}$ ball. Specifically, if the current iterate is in the interior of the feasible set, then the weighted $\ell_{1}$ ball is formed by linearizing the $\ell_{p}$ norm at the current iterate. If the current iterate is on the boundary of the feasible set, then the weighted $\ell_{1}$ ball is formed differently by keeping those zero components in the current iterate still zero. In our analysis, we prove that the generated iterates converge to a first-order stationary point. Numerical experiments demonstrate the effectiveness of the proposed method.

An Iteratively Reweighted Method for Sparse Optimization on Nonconvex $\ell_{p}$ Ball

TL;DR

An iteratively reweighted method is proposed, which solves a sequence of weighted -ball projection subproblems, and it is proved that the generated iterates converge to a first-order stationary point.

Abstract

This paper is intended to solve the nonconvex -ball constrained nonlinear optimization problems. An iteratively reweighted method is proposed, which solves a sequence of weighted -ball projection subproblems. At each iteration, the next iterate is obtained by moving along the negative gradient with a stepsize and then projecting the resulted point onto the weighted ball to approximate the ball. Specifically, if the current iterate is in the interior of the feasible set, then the weighted ball is formed by linearizing the norm at the current iterate. If the current iterate is on the boundary of the feasible set, then the weighted ball is formed differently by keeping those zero components in the current iterate still zero. In our analysis, we prove that the generated iterates converge to a first-order stationary point. Numerical experiments demonstrate the effectiveness of the proposed method.

Paper Structure

This paper contains 12 sections, 8 theorems, 46 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Algorithm description
  4. Convergence analysis
  5. First-order optimality condition
  6. Well-posedness
  7. Global convergence
  8. Numerical Experiments
  9. $\ell_{p}$-constrained Least squares on synthetic data
  10. $\ell_{p}$-constrained logistic regression on real-world data
  11. Conclusions
  12. Appendix: Proof of \ref{['thm:FirstOrderNecessaryCondition']}: Characterizing the elements of the Fréchet normal cone

Key Result

Lemma 4.2

If $\bar{\bm{x}} \in \mathbb{R}^n$ satisfies $\|\bar{\bm{x}}\|_p^p< r$, then ${\mathcal{N}}_\Omega(\bar{\bm{x}}) = \{\bm{0}\}$. If $\bar{\bm{x}} \in \mathbb{R}^n$ satisfies $\|\bar{\bm{x}}\|_p^p= r$, then $\bm{\eta}\in{\mathcal{N}}_{\Omega}(\bar{\bm{x}})$ with

Figures (2)

  • Figure 1: The empirical probability of success versus $m$ for various $\ell_{p}$ ball constraints.
  • Figure 2: The empirical probability of success versus $m$ for $\ell_{0.5}$ ball and $\ell_1$ ball constraints.

Theorems & Definitions (15)

  • Definition 4.1
  • Lemma 4.2
  • Theorem 4.3: Fermat's rule
  • Lemma 4.4
  • proof
  • Lemma 4.5
  • proof
  • Theorem 4.6
  • Lemma 4.7
  • proof
  • ...and 5 more