Subspace Newton's Method for $\ell_0$-Regularized Optimization Problems with Box Constraint
Yuge Ye, Qingna Li
TL;DR
The paper tackles box-constrained $\ell_0$-regularized sparse optimization by introducing a $\tau$-stationary point and a subspace Newton method (BNL0R) that updates inactive components via Newton steps while updating active components with proximal gradient steps. It proves global convergence and local quadratic convergence under mild smoothness/convexity assumptions and demonstrates the method’s effectiveness through extensive compressed sensing and image-recovery experiments, where BN L0R outperforms first-order methods in iterations, time, and accuracy. The approach leverages an active/inactive index partition, an Armijo line search for descent, and a fallback projection-gradient step to ensure robust progress toward a $\tau$-stationary point or a local/global minimizer. The results establish a practical second-order framework for nonconvex, box-constrained $\ell_0$-regularized problems with strong potential for sparse recovery and signal processing applications.
Abstract
This paper investigates the box-constrained $\ell_0$-regularized sparse optimization problem. We introduce the concept of a $τ$-stationary point and establish its connection to the local and global minima of the box-constrained $\ell_0$-regularized sparse optimization problem. We utilize the $τ$-stationary points to define the support set, which we divide into active and inactive components. Subsequently, the Newton's method is employed to update the non-active variables, while the proximal gradient method is utilized to update the active variables. If the Newton's method fails, we use the proximal gradient step to update all variables. Under some mild conditions, we prove the global convergence and the local quadratic convergence rate. Finally, experimental results demonstrate the efficiency of our method.