Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards An Efficient Approach for the Nonconvex $\ell_p$ Ball Projection: Algorithm and Analysis

Xiangyu Yang, Jiashan Wang, Hao Wang

TL;DR

A novel numerical approach for computing the stationary point through solving a sequence of projections onto the reweighted `1-balls is developed, which is practically simple to implement and computationally efficient.

Abstract

This paper primarily focuses on computing the Euclidean projection of a vector onto the $\ell_{p}$ ball in which $p\in(0,1)$. Such a problem emerges as the core building block in statistical machine learning and signal processing tasks because of its ability to promote the sparsity of the desired solution. However, efficient numerical algorithms for finding the projections are still not available, particularly in large-scale optimization. To meet this challenge, we first derive the first-order necessary optimality conditions of this problem. Based on this characterization, we develop a novel numerical approach for computing the stationary point by solving a sequence of projections onto the reweighted $\ell_{1}$-balls. This method is practically simple to implement and computationally efficient. Moreover, the proposed algorithm is shown to converge uniquely under mild conditions and has a worst-case $O(1/\sqrt{k})$ convergence rate. Numerical experiments demonstrate the efficiency of our proposed algorithm.

Towards An Efficient Approach for the Nonconvex $\ell_p$ Ball Projection: Algorithm and Analysis

TL;DR

A novel numerical approach for computing the stationary point through solving a sequence of projections onto the reweighted `1-balls is developed, which is practically simple to implement and computationally efficient.

Abstract

This paper primarily focuses on computing the Euclidean projection of a vector onto the ball in which . Such a problem emerges as the core building block in statistical machine learning and signal processing tasks because of its ability to promote the sparsity of the desired solution. However, efficient numerical algorithms for finding the projections are still not available, particularly in large-scale optimization. To meet this challenge, we first derive the first-order necessary optimality conditions of this problem. Based on this characterization, we develop a novel numerical approach for computing the stationary point by solving a sequence of projections onto the reweighted -balls. This method is practically simple to implement and computationally efficient. Moreover, the proposed algorithm is shown to converge uniquely under mild conditions and has a worst-case convergence rate. Numerical experiments demonstrate the efficiency of our proposed algorithm.

Paper Structure

This paper contains 29 sections, 20 theorems, 74 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Organization
  3. Notation and Preliminaries
  4. Proposed Algorithm
  5. First-order Necessary Conditions for Optimality
  6. An Iteratively Reweighted $\ell_1$ Ball Projection Algorithm
  7. Weighted $\ell_{1}$-ball Approximation
  8. A Dynamic Updating Strategy for $\epsilon$
  9. Convergence Analysis of IRBP
  10. Well-posedness and Basic Properties
  11. Convergence Results
  12. Uniqueness of the Cluster Points
  13. Local Complexity Analysis
  14. Worst-case Complexity Analysis
  15. Numerical Experiments
  16. ...and 14 more sections

Key Result

Lemma 2.1

Let $\bm{y}$ be a point satisfying $\sum_{i=1}^n w_i |y_i|^p > \gamma$ and $\bm{x}^*$ be a global optimal solution of general.projection. Then, the following properties hold:

Figures (4)

  • Figure 1: A two-dimensional illustration of the normal cone for the unit $\ell_{0.5}$-ball constraint in the nonnegative orthant. The normal cone has infinitely many normal directions at $(0,1)$ and $(1,0)$ as represented in the shaded region, and it has a single normal direction besides those two points.
  • Figure 2: The iteration path of projecting $\bm{y} = \left[0.5, 0.45\right]^{T}$ onto the unit $\ell_{0.5}$-ball. In particular, the solid circles denote the iterates with the randomly generated $\bm{\epsilon}^{0} = \left[0.072, 0.463\right]^{T}$. The dashed line represents the boundary of a weighted $\ell_1$ ball constructed at each subproblem.
  • Figure 3: Performance profiles for IRBP and RS with $p \in \{0.4,0.8\}$ and $n = 10^{2}$.
  • Figure 4: Box plots of the elapsed wall-clock time (seconds) for IRBP with $p \in \{0.4, 0.8\}$ and $n \in \{10^{3}, 10^{4}, 10^{5}, 10^{6}\}$. Each presented elapsed wall-clock time value is averaged over $20$ runs.

Theorems & Definitions (39)

  • Definition 1.1
  • Lemma 2.1
  • proof
  • Theorem 2.2
  • Proposition 2.3
  • Theorem 4.1: Well-posedness
  • proof
  • Lemma 4.2: Descent property
  • proof
  • Lemma 4.3
  • ...and 29 more