Nonsmooth Newton methods with effective subspaces for polyhedral regularization

TL;DR

The paper tackles convex composite optimization of the form $\min_x f(x)+g(x)$ where $f$ is smooth and $g$ is polyhedral. It introduces Newton-type methods that avoid computing second-order information for $g$ by restricting Newton steps to the effective subspace $L_k=\mathrm{par}\,\partial g^*(z_k)$, and proves local quadratic convergence under tilt-stability. A key contribution is the explicit computation of these effective subspaces for common polyhedral regularizers such as $\ell_1$, $\ell_\infty$, sorted $\ell_1$ (SLOPE), and 1D total variation, enabling scalable implementations that leverage first-order data. The paper demonstrates substantial acceleration over ISTA/FISTA and competitive performance against other second-order schemes in tasks including Lasso, OSCAR, TV, and Poisson-based image super-resolution, highlighting the practical impact of tilt-stability-guided Newton steps in nonsmooth convex optimization.

Abstract

We propose several new nonsmooth Newton methods for solving convex composite optimization problems with polyhedral regularizers, while avoiding the computation of complicated second-order information on these functions. Under the tilt-stability condition at the optimal solution, these methods achieve the quadratic convergence rates expected of Newton schemes. Numerical experiments on Lasso, generalized Lasso, OSCAR-regularized least-square problems, and an image super-resolution task illustrate both the broad applicability and the accelerated convergence profile of the proposed algorithms, in comparison with first-order and several recently developed nonsmooth Newton schemes.

Figures (6)

• Figure 1: $\ell_1$ norm
• Figure 2: $\ell_\infty$ norm
• Figure 3: OSCAR regularizer
• Figure 4: 1D total variation semi-norm
• Figure 5: Images in SMLM ISBI 2013 dataset

Theorems & Definitions (23)

• Definition 2.1: Tilt stability
• Theorem 2.2
• Theorem 3.1: Quadratic convergence of the Newton method with effective subspaces
• proof
• Remark 3.2
• Corollary 3.3: Quadratic convergence of the ISTA-Newton method with effective subspaces
• proof
• Remark 3.4
• Proposition 3.5
• proof