Modified Block Newton Algorithm for $\ell_0$-Regularized Optimization
Yuge Ye, Qingna Li
TL;DR
This work tackles $\ell_0$-regularized sparse optimization by developing a globally convergent Newton-type method that reduces Jacobian computations to a block-diagonal form and adds regularization to handle singularities. The Modified Block Newton Method (MBNL0R) updates directions using $Q_k = \nabla_{T_k}^2 f(x^k) + \mu_k I$ with a selective block structure, ensuring the iterates remain sparse and converge via a line search. The authors prove global convergence and, under standard smoothness/convexity assumptions plus Lipschitz Hessian, local quadratic convergence, with the index set $T_k$ stabilizing at a finite step. Numerical experiments across compressed sensing, 2-D imaging, and sparse linear complementarity problems show MBNL0R often matches or surpasses NL0R in speed while maintaining high accuracy, highlighting its practicality for large-scale sparse optimization problems.
Abstract
In this paper, we propose a globally convergent Newton type method to solve $\ell_0$ regularized sparse optimization problem. In fact, a line search strategy is applied to the Newton method to obtain global convergence. The Jacobian matrix of the original problem is a block upper triangular matrix. To reduce the computational burden, our method only requires the calculation of the block diagonal. We also introduced regularization to overcome matrix singularity. Although we only use the block-diagonal part of the Jacobian matrix, our algorithm still maintains global convergence and achieves a local quadratic convergence rate. Numerical results demonstrate the efficiency of our method.